Framework integrations

eval-ai-library ships callback adapters for the major agent frameworks. Each one turns framework events into the same span shape the raw SDK produces, so the Runtime eval pages look the same whatever you built on. You still call start_trace / set_trace_metadata / end_trace around the invocation; the callback fills in the spans.

Claude Agent SDK / Anthropic Messages API

Ships an explicit trace collector that turns Anthropic content blocks (text, thinking, tool_use) into spans. Works with both the raw API and claude_agent_sdk streams.

from eval_lib.tracing import tracer
from eval_lib.tracing.claude_agent_callback import ClaudeAgentTraceCollector

collector = ClaudeAgentTraceCollector()
tracer.start_trace("claude_agent")
tracer.set_trace_metadata(session_id=session_id, user_id=user_id)

# Option A: raw Anthropic Messages API
response = client.messages.create(model="claude-opus-4-8", ...)
collector.process_response(response)

# Option B: claude_agent_sdk stream
async for msg in client.receive_messages():
    collector.process_sdk_message(msg)

tracer.end_trace()

LangChain

Register the callback handler on your chain or agent and every LLM and tool call inside it emits a span. No manual span bookkeeping.

from eval_lib.tracing import tracer
from eval_lib.tracing.langchain_callback import EvalLibCallbackHandler

tracer.start_trace("langchain_agent")
tracer.set_trace_metadata(session_id=session_id, user_id=user_id)

handler = EvalLibCallbackHandler()
result = my_chain.invoke(
    {"question": user_message},
    config={"callbacks": [handler]},
)
tracer.end_trace()

Other frameworks

Same pattern: import the callback or collector from the matching sub-module and wire it into the framework's callback slot.

Framework	Module
AutoGen	eval_lib.tracing.autogen_callback
CrewAI	eval_lib.tracing.crewai_callback
Haystack	eval_lib.tracing.haystack_callback
LlamaIndex	eval_lib.tracing.llamaindex_callback
OpenAI Assistants	eval_lib.tracing.openai_assistants_callback
OpenAI Responses API	eval_lib.tracing.openai_responses_callback
Phidata	eval_lib.tracing.phidata_callback
Semantic Kernel	eval_lib.tracing.semantic_kernel_callback
smolagents	eval_lib.tracing.smolagents_callback

Already on OpenTelemetry inside Python?

Two options, and they're not the same thing:

• EvalLibSpanExporter in eval_lib.tracing.otel_collector is an OpenTelemetry SpanExporter. Register it on your tracer provider and OTel spans are converted into eval-lib spans in-process and shipped by the SDK. Use it when you want the SDK's redaction, retries, and set_trace_metadata fields on top of OTel instrumentation.
• The OTLP endpoint needs no Python at all. Point a stock OTLP/HTTP exporter at EvaliQA and you're done. See OpenTelemetry.

If you're unsure, take the OTLP endpoint. It's four env vars and it survives a language change.

Mixing frameworks in one trace

A LangChain orchestrator that spawns a Claude Agent SDK sub-agent is fine. Register both callbacks inside the same start_trace / end_trace window and the spans from each land in one tree, nested where they happened.

Let EvaliQA open the pull request

If your agent lives on GitHub, the setup guide can do the wiring for you. Connect GitHub, pick the project, and EvaliQA reads the repository, plans the smallest instrumentation diff it can using your workspace's Platform AI agent, and opens a pull request for you to review. Nothing is merged without your approval. Prefer to stay off GitHub? The coding-agent prompt tab gives the same instructions to your local Cursor, Claude Code, or Windsurf session.

Related

• SDK setup
• Verify and troubleshoot