OpenTelemetry

The second door. If your application already emits OpenTelemetry spans, add one more OTLP/HTTP exporter and EvaliQA receives the same traces your Collector, Datadog, or Azure Monitor already see. Nothing is installed, eval-ai-library is not involved, and your existing exporters keep working.

This is the door for stacks the Python SDK cannot reach: Amazon Bedrock Agents, Azure AI Foundry, a TypeScript service on the Vercel AI SDK, n8n, Make, LangGraph Cloud, or anything else that speaks OTLP.

1. Environment variables

Set these where your OpenTelemetry SDK or Collector runs:

OTEL_EXPORTER_OTLP_ENDPOINT=https://<your-evaliqa-host>/api/traces/otlp
OTEL_EXPORTER_OTLP_HEADERS=x-api-key=evx_...
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_RESOURCE_ATTRIBUTES=service.name=my-agent

• OTEL_EXPORTER_OTLP_ENDPOINT stops at /otlp. Exporters append /v1/traces themselves; the full route is POST /api/traces/otlp/v1/traces.
• OTEL_EXPORTER_OTLP_HEADERS carries the API key as x-api-key. Same evx_ key as the SDK, created on the project's Tracing tab. The key is bound to a project, so no project id travels with the spans.
• OTEL_EXPORTER_OTLP_PROTOCOL can be http/protobuf (the exporter default) or http/json. Both are accepted, gzip too. gRPC is not.
• OTEL_RESOURCE_ATTRIBUTES names your service. EvaliQA keeps it in the trace metadata as service_name.

The OpenTelemetry tab of the setup guide on the Runtime eval pages renders this block with your host and key filled in.

2. Point your exporter at EvaliQA

Plain OpenTelemetry SDK

With the env vars set, a stock OTLP/HTTP exporter needs no arguments:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)

Then instrument as you already do: OpenLLMetry, OpenInference, the official opentelemetry-instrumentation-* packages for OpenAI, Anthropic, Bedrock, or your own manual spans. EvaliQA reads all of their attribute conventions, see What EvaliQA reads.

OpenTelemetry Collector

Running a Collector in between? Add EvaliQA as one more exporter and keep the ones you have:

exporters:
  otlphttp/evaliqa:
    endpoint: https://<your-evaliqa-host>/api/traces/otlp
    headers:
      x-api-key: evx_...
service:
  pipelines:
    traces:
      exporters: [otlphttp/evaliqa]   # keep your existing exporters in this list

Amazon Bedrock Agents

The ADOT collector that ships with Application Signals is a stock OpenTelemetry Collector. Either add the otlphttp/evaliqa exporter above to its traces pipeline, or set the four env vars on the Lambda or ECS task that runs the ADOT SDK. Bedrock models price correctly only when the gen_ai.system attribute is present, see Cost.

Azure AI Foundry

Keep configure_azure_monitor() as it is and add a second span processor to the global tracer provider:

from azure.monitor.opentelemetry import configure_azure_monitor
from opentelemetry import trace
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

configure_azure_monitor()
trace.get_tracer_provider().add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter())
)

Azure Monitor keeps receiving everything; EvaliQA receives a copy.

Everything else

n8n, Make, LangGraph Cloud, a Node service on the Vercel AI SDK: if it can emit OTLP/HTTP, the four env vars are the whole integration.

How spans become one trace

OpenTelemetry exporters ship spans as they end, in batches. The root span of a request ends last, and often lands in a later HTTP request than its children. EvaliQA's feed shows one row per trace, not one per batch, so incoming spans are parked per trace_id until the trace is complete:

• A batch that contains the root span (no parent) completes the trace: it's assembled and stored at once, and appears on the Traces tab within seconds.
• A trace whose root never arrives, because a parent span was sampled out or lives in a service that isn't exporting, is stored after 20 seconds of quiet. Each new batch for that trace pushes the timer forward, so a long agent is never cut in half.
• Retried batches are harmless: spans are keyed by span id, so a duplicate overwrites itself.
• Spans with a malformed trace or span id are counted in the response's partialSuccess.rejectedSpans; the rest of the batch is kept.

If the buffer is unavailable EvaliQA answers 503 with Retry-After: 5 and the exporter's built-in retry re-sends the batch. Nothing is acknowledged that wasn't stored.

What EvaliQA reads from your spans

The mapper tries the newest OpenTelemetry GenAI semantic conventions first, then the older spellings from OpenLLMetry, OpenInference, and the Vercel AI SDK. You don't have to rename anything.

Field	Attributes, in order of preference
Model	gen_ai.response.model, gen_ai.request.model, llm.response.model, llm.request.model, llm.model_name, ai.model.id, model
Input tokens	gen_ai.usage.input_tokens, gen_ai.usage.prompt_tokens, llm.usage.prompt_tokens, llm.token_count.prompt, ai.usage.promptTokens, input_tokens
Output tokens	gen_ai.usage.output_tokens, gen_ai.usage.completion_tokens, llm.usage.completion_tokens, llm.token_count.completion, ai.usage.completionTokens, output_tokens
Cost	gen_ai.usage.cost, gen_ai.usage.cost_usd, gen_ai.cost.usd, llm.usage.cost, cost_usd
Provider	gen_ai.provider.name, gen_ai.system, llm.system, llm.vendor, ai.model.provider
Tool name	gen_ai.tool.name, tool.name, tool_name, ai.toolCall.name
Session	gen_ai.conversation.id, session.id, conversation.id, thread.id, evaliqa.session_id, langfuse.session.id, session_id
User	enduser.id, user.id, evaliqa.user_id, langfuse.user.id, user_id
Messages	gen_ai.input.messages, gen_ai.prompt, input.value, traceloop.entity.input and their output counterparts; indexed gen_ai.prompt.N.content; OpenInference llm.input_messages.N.message.content; span events (gen_ai.user.message, gen_ai.assistant.message, gen_ai.choice, gen_ai.content.prompt)

The trace-level input and output, the fields online evaluation scores by default, are filled from those messages.

Span types

Each span gets one of the same types the SDK uses (llm_call, tool_call, agent_step, reasoning, retrieval, evaluation, custom), decided in this order:

1. gen_ai.operation.name: chat, text_completion, generate_content, embeddings → llm_call; execute_tool → tool_call; invoke_agent, create_agent → agent_step.
2. openinference.span.kind, traceloop.span.kind, confident.span.type, or the Vercel AI SDK's ai.operationId.
3. Attribute presence: a model attribute makes it an LLM call, a tool-name attribute makes it a tool call.
4. An inbound HTTP or RPC span (http.route, or kind SERVER) is an agent_step, so POST /chat is never mistaken for a model call.
5. Name heuristics, then custom.

If your timeline is all custom, your instrumentation isn't setting any of the attributes above. Add gen_ai.operation.name and it sorts itself out.

Cost: reported or estimated

OpenTelemetry carries tokens far more often than it carries cost. When a span has a cost attribute, EvaliQA sums it and tags the trace cost_source: reported. When it only has tokens, EvaliQA prices them from the model catalogue the rest of the platform uses and tags the trace cost_source: estimated. The feed always shows which one you're looking at; an estimate is never presented as an invoice.

Bedrock, Azure, Vertex, Gemini, Groq, Mistral, DeepSeek, and similar hosts price the same model name differently, so for them the price lookup needs the provider. That's what gen_ai.system is for. Without it the trace still lands, with no cost.

Good to know

• service.name names the project only when you authenticate with a platform JWT instead of an API key. With a key, the key's project wins.
• Every OTLP trace is stamped source: otlp in its metadata, alongside the resource attributes, the instrumentation scope, and the root span name, so you can tell the two doors apart in the trace view.
• Two self-hosted settings control the buffer: OTLP_TRACE_QUIET_WINDOW_SEC (default 20) and OTLP_TRACE_BUFFER_TTL_SEC (default 3600). Hosted workspaces use the defaults.

Related

• Sessions and users, which attributes group traces into sessions and users.
• Verify and troubleshoot, a curl smoke test for the OTLP door.
• Runtime tracing, the SDK door and the shared prerequisites.