You open your tracing UI to debug a slow chat completion. The app trace shows a 4.2 second request with one opaque child span called "llm call." You open a second tab for the LiteLLM proxy and find a trace that looks about right, but nothing ties the two together. Different trace IDs, no shared parent, no way to say which gateway request belongs to which user action.
Most LiteLLM tracing guides stop at turning a callback on. That gets you spans, and it does not get you one trace. The harder part is making the app request and the proxy request land in the same trace with the right parent-child links.
This post covers that part: where the split comes from, how to propagate W3C traceparent from your app to the proxy, and how to check the result with trace IDs and parent span IDs. One caveat up front: I wrote this from the LiteLLM and OpenTelemetry docs, not from a hands-on test rig. Treat the settings below as things to verify in your own deployment. Trust the verification steps most.
Why app traces and gateway traces split apart
A trace is a set of spans that share one trace ID. Spans point at their parent with a parent span ID. That is the whole mechanism. When your app and the LiteLLM proxy produce two unlinked entries, one of two things happened: the proxy started a fresh trace because it received no trace context, or it received context and ignored it.
The usual cause is the first one. Your app creates a span, calls the proxy over HTTP, and sends no traceparent header. The proxy has nothing to continue, so it mints a new trace ID and treats the request as a root. Both sides are instrumented and both are "working." They just can't see each other.
This isn't a LiteLLM bug. It's how distributed tracing behaves when context doesn't cross a process boundary. A model call through a gateway is a network hop like any other, and it needs the same context propagation you would add between two microservices.
The cost shows up when you debug. A slow request could be your retrieval step, your prompt assembly, the proxy's routing and retry logic, or the provider. With split traces you can see pieces of the answer in different places and no way to line them up except by timestamps and guessing. Timestamps lie under load.
The fix is boring on purpose. Send the W3C traceparent header from the app, have the proxy honor it, and confirm the result. Everything else in this post is a variation on those three steps.
Three kinds of spans: SDK calls, proxy spans, and application tool spans
Before you wire anything, know which spans you expect to see. A LiteLLM setup has three distinct producers, and mixing them up is the most common reason people think correlation is broken when it isn't.
SDK call spans. If you use the LiteLLM Python SDK directly inside your app, the SDK emits spans for each completion call in your process. These live in the same process as your app code, so context propagation is automatic through the OpenTelemetry context. No headers involved.
Proxy spans. If you run the LiteLLM proxy as a separate service, the proxy emits its own spans for each request it handles: routing, the upstream provider call, and so on. These live in a different process, which is exactly why header propagation matters.
Application tool spans. These are spans you create yourself around tool calls, retrievals, and business logic. LiteLLM knows nothing about them. They exist only if you instrument them.
A healthy end-to-end trace has all three in a clear shape: your request span at the root, tool and retrieval spans as siblings or children, and the LiteLLM call (SDK or proxy) nested under the step that triggered it. If you use both the SDK and the proxy, expect an SDK span in your app that parents a proxy span in the gateway.
Write down the tree you expect before you test. A concrete target beats "it looks connected." For example: handle_request > retrieve_docs and handle_request > litellm completion > proxy request span > provider call. When your actual tree differs, you know exactly which edge is missing.
LiteLLM OpenTelemetry v1 vs v2: check which callback you are running
LiteLLM has shipped more than one OpenTelemetry integration path, and the callback name in your config decides which one you get. Check this first, because the two can differ in span shape and in how they treat incoming trace context.
Pull the version you're actually running and read its docs instead of trusting a blog post (this one included). In the proxy, the callback is declared in config.yaml under litellm_settings. In the SDK, it's set through litellm.callbacks or the success and failure callback lists. Search your config and code for otel and note the exact string.
litellm_settings:
callbacks: ["otel"]Then look at what the LiteLLM docs for your installed version say that name does. Answer three questions from the docs, not from memory:
Does this callback create a span per request, and what are its attribute names?
Does it read an incoming
traceparentand use it as the parent?Is there a separate setting that controls whether the proxy trusts incoming context?
The third question matters more than it looks. Some gateway deployments deliberately ignore inbound trace context for requests from untrusted callers, so outside clients can't inject themselves into your traces. If your version exposes such a setting, it may be the one standing between you and a connected trace. Section six covers where to look.
Pin the version in your lockfile or image tag while you debug. If you upgrade mid-investigation and the callback behavior shifts, you'll chase a ghost. Record the version next to your findings so the next engineer can reproduce them.
Propagating W3C traceparent from your app to the LiteLLM proxy
The W3C Trace Context spec defines the traceparent header. It carries four dash-separated fields: version, trace ID, parent span ID, and trace flags. A typical value looks like 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01. The proxy reads the trace ID and parent span ID from it and, if configured to honor it, continues your trace. The W3C publishes the spec as Trace Context.
In practice you rarely write this header by hand. OpenTelemetry's propagator injects it when you make an instrumented HTTP call. In Python, if you call the proxy with an HTTP client that has OpenTelemetry instrumentation enabled (the requests or httpx instrumentors, for example), the header is added from the active span automatically. In TypeScript, the Node HTTP or fetch instrumentation does the same.
If you call the proxy through the OpenAI client pointed at your proxy base URL, check that your HTTP layer is instrumented. The OpenAI client uses an HTTP library underneath, and if that library isn't instrumented, no header goes out.
When you can't rely on auto-instrumentation, inject it explicitly:
from opentelemetry.propagate import inject
headers = {}
inject(headers) # adds traceparent from the current span
# pass headers as default_headers or extra_headers to your clientThen confirm the header is actually leaving your process. Don't assume. Log the outgoing headers once in a dev environment, or capture one request with a proxy tool. A missing traceparent at this stage explains most correlation failures, and the check takes thirty seconds and saves an afternoon.
Verify correlation: matching trace IDs and parent-child span relationships
Correlation is a claim you can test. Don't eyeball two dashboards and decide they look related. Run a single request and check three things.
1. Same trace ID. Make one request from your app and note the active trace ID in your app logs. In the backend, find the proxy's span for that request. Its trace ID must equal the app's. If they differ, propagation failed and nothing else matters yet.
2. Correct parent span ID. The proxy's top span should name a parent span ID that matches the span your app was in when it made the call. That's the span whose ID was sent in the traceparent header. A shared trace ID with a missing or wrong parent produces an orphaned span: same trace, wrong place in the tree.
3. Expected tree shape. Compare the rendered tree with the target you wrote in section two. The proxy span should sit under the LiteLLM call span or the step that made the request, not float at the root.
Here is a quick way to log the first two from the app side:
from opentelemetry import trace
ctx = trace.get_current_span().get_span_context()
print(format(ctx.trace_id, "032x"), format(ctx.span_id, "016x"))Compare those values to the proxy span's trace ID and parent span ID in your backend. They should line up exactly: your trace ID equals the proxy's trace ID, and your span ID equals the proxy span's parent span ID.
Repeat with a failing request, such as a bad model name that makes the upstream call error. Error paths often skip instrumentation that the happy path has, and an error trace you can't connect to its caller is the one you needed most. Check both.
When correlation breaks: context propagation settings, missing headers, and collector hops
When the trace IDs don't match, work backward along the path the header takes. It can die in four places, and each has a distinct signature.
The app never sent it. Symptom: proxy spans have a fresh trace ID and no parent. Check that your HTTP client is instrumented, or inject manually as shown earlier. Confirm by logging outgoing headers.
Something between the app and proxy stripped it. Load balancers, API gateways, and ingress layers sometimes drop unknown headers or rewrite them. Symptom: the app sends it, the proxy never receives it. Log incoming headers at the proxy, or call the proxy directly and bypass the layers in between. If direct calls correlate and routed calls don't, a hop is eating the header.
The proxy received it and ignored it. Symptom: the header arrives, the trace still splits. Look in the LiteLLM docs for your version for a setting that controls whether incoming trace context is honored. Gateways commonly gate this for safety, and the default may be to start a fresh trace. Read the docs here instead of guessing, because the setting name and default depend on your version.
The collector or exporter broke the link. Symptom: both sides export correctly but the backend shows broken trees. If you run an OpenTelemetry Collector between services and your backend, check that no processor rewrites trace or span IDs, and that both services export to the same backend. If the Collector authenticates upstream using secrets, see OpenTelemetry Collector environment variables for auth for a clean way to keep keys out of config files.
Watch sampling too. If the app samples a trace in and the proxy samples independently with a different policy, you can end up with half a trace. Honor the sampled flag in the traceparent flags field, or use a parent-based sampler on both sides.
Sending the correlated traces to telemetry.dev over OTLP/HTTP
Once app and proxy spans share a trace, both sets need to reach the same backend. telemetry.dev accepts standard OTLP/HTTP, so any OpenTelemetry exporter or Collector can send to it: the LiteLLM proxy's OpenTelemetry callback, your Python or TypeScript app, and a Collector sitting in between. The transport has no special LiteLLM-only path to learn.
The LiteLLM OpenTelemetry integration reads the standard OpenTelemetry environment variables for its exporter. Set the endpoint and headers on the proxy, and set the same destination on your app, so both sides write to one workspace:
OTEL_EXPORTER_OTLP_ENDPOINT=<your telemetry.dev OTLP/HTTP endpoint>
OTEL_EXPORTER_OTLP_HEADERS=<auth header from your project settings>
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobufGet the exact endpoint and header values from the telemetry.dev OTLP reference and the LiteLLM integration docs. I'm not reproducing them here from memory. telemetry.dev also provides first-party TypeScript and Python SDKs and a LiteLLM integration for Python if you'd rather not hand-configure exporters.
Correlating app and gateway spans means each request now produces spans from both sides, so a trace costs more units than the app alone. The Free plan includes 30k units, which is enough to run the verification steps from this post before you decide on anything larger.
After the first correlated request lands, repeat the checks from section five in the backend itself: one trace, proxy span nested under the app span, correct parent ID. If you also capture prompts and outputs, read OpenTelemetry GenAI prompt capture: inspect before export first. Capture is on by default, and you'll want to decide what leaves your app.
Conclusion
Correlation isn't a setting you turn on and trust. It's a property you check: same trace ID, correct parent span ID, expected tree shape. Do that once per environment, on a success and on a failure, and write the result down.
Here is a concrete next step. Point both your app and your LiteLLM proxy at one telemetry.dev workspace over OTLP/HTTP, send a single request with traceparent injected, and compare the proxy span's parent ID to your app's span ID. The Free plan (30k units) covers that experiment. If the IDs match, you have one trace where you used to have two. If they don't, section six tells you which hop to look at first.
Share this article
