SDK setup

The SDK door. eval-ai-library wraps your agent's entry point, records each step as a span, and POSTs the finished trace to EvaliQA as JSON. If your stack already emits OpenTelemetry, skip this page and read OpenTelemetry instead.

Before you start

You need a project, a tracing API key, and your EvaliQA host. The Runtime tracing page covers all three. The short version: Project → Tracing tab → New API key, copy the evx_… value while it's on screen.

1. Install

Add the tracing extra to your agent's dependencies. Python 3.9 or newer.

pip install "eval-ai-library[tracing]"

If eval-ai-library is already there for its metric library, upgrade it. Session and user grouping, cached-token accounting, and the cost fields all arrived in the 0.7 line; the current release is 0.7.24 and the platform itself runs against the latest one.

2. Configure environment variables

TRACING_ENABLED=true
TRACING_URL=https://<your-evaliqa-host>/api/traces/ingest
TRACING_PROJECT=<your-project-uuid>
TRACING_API_KEY=evx_...

• TRACING_ENABLED: the kill switch. Defaults to false, so a forgotten var means no network calls at all. Leave it off in local dev if you don't want dev traffic in the feed.
• TRACING_URL: the ingest endpoint, your host plus /api/traces/ingest.
• TRACING_PROJECT: the project UUID. The API key already pins the project, so with a key this value is informational; it becomes the project name only when you authenticate with a platform JWT instead of a key. Set it anyway, the SDK otherwise defaults it to the string default.
• TRACING_API_KEY: the evx_… value. Secrets store, not git.

Optional knobs, all read from the environment:

Variable	Default	What it does
TRACING_STRICT	false	Raise on a failed send instead of logging it. Turn on in CI so a bad key fails the pipeline.
TRACING_REDACT	true	Scrub credential-looking keys and values before the trace leaves the process. Leave on; a captured client object carries its API key.
TRACING_MAX_RETRIES	2	Retries for a failed POST, so three attempts total. Only network errors, timeouts, 5xx, 408, and 429 are retried; a 4xx fails fast. 0 disables retrying.
TRACING_RETRY_BACKOFF	0.5	Seconds before the first retry, doubled each attempt.
TRACING_MAX_FIELD_LENGTH	unset	Cap on characters per captured span field. Unset keeps fields whole; a cap marks the truncation explicitly.
TRACING_SINK	http	memory or file for tests. file appends JSONL to TRACING_SINK_PATH (default traces.jsonl).

3. Basic usage

Wrap your agent's entry point in start_trace + end_trace. Each meaningful step becomes a tracer.trace(...) context manager, and set_trace_metadata fills in the trace-level fields before the trace ships.

from eval_lib.tracing import tracer

def my_agent(user_message: str, session_id: str, user_id: str) -> str:
    tracer.start_trace("my_agent")
    try:
        with tracer.trace("plan", span_type="reasoning"):
            plan = build_plan(user_message)

        with tracer.trace("search", span_type="tool_call"):
            docs = search(plan.query)

        with tracer.trace("answer", span_type="llm_call"):
            answer, usage = call_model(plan, docs)

        tracer.set_trace_metadata(
            model="gpt-4o-mini",
            input=user_message,
            output=answer,
            input_tokens=usage.input_tokens,
            output_tokens=usage.output_tokens,
            session_id=session_id,
            user_id=user_id,
        )
        return answer
    finally:
        tracer.end_trace()

end_trace() in a finally block matters: if the agent raises, the trace still ships, with the span that raised marked as an error, and the error badge on the Traces tab is how you'll notice.

Trace metadata worth setting

set_trace_metadata accepts more than the example shows. The fields EvaliQA uses directly:

• model, input, output: what the trace page shows first, and what online evaluation scores by default.
• input_tokens, output_tokens, total_tokens, plus cached_tokens and reasoning_tokens when your provider reports them. Cached tokens are billed at a fraction of the input rate; in a multi-agent loop that resends the whole history each turn, ignoring them overstates cost several-fold.
• cost_usd and cost_source. Pass cost_usd when your SDK reports an authoritative number (Claude Agent SDK's total_cost_usd, for instance) and set cost_source="reported". Leave both out and the feed shows what the model and tokens imply, tagged estimated.
• response_time: seconds, end to end.
• session_id, user_id: see Sessions and users.

Decorators instead of context managers

For code you'd rather not restructure, three decorators wrap a function in a span of the matching type and capture its arguments and return value:

from eval_lib.tracing import trace_llm, trace_tool, trace_step

@trace_llm(name="chat_completion")
async def get_completion(prompt: str) -> str: ...

@trace_tool(name="web_search")
async def search_web(query: str) -> list[dict]: ...

@trace_step(name="reason")
def reason_about(input_data) -> str: ...

capture_input=False / capture_output=False keep a payload out of the trace when it's large or sensitive. You still need start_trace / end_trace around the whole invocation.

Span types

Pass span_type= so the timeline colours and icons each step correctly. tracer.trace() defaults to agent_step.

• llm_call: a call to a model.
• tool_call: an external tool or function call.
• agent_step: a higher-level step in the plan.
• reasoning: planning, thinking.
• retrieval: vector or knowledge lookup.
• evaluation: the agent checking its own result.
• custom: anything else.

What's next

• Verify and troubleshoot, fire a smoke trace and confirm it lands.
• Sessions and users, grouping and attribution.
• Framework integrations, callbacks that do the span bookkeeping for you.