LiteLLM

Trace LiteLLM completion, Responses API, embedding, rerank, stream, and Router calls from Python.

Call instrument_litellm() once, and each LiteLLM call becomes a span in telemetry.dev, whichever model provider LiteLLM sends it to. Your calls and their return values don’t change. The integration is for Python.

What you get

• Chat messages, Responses API input, models, sampling parameters, response formats, embedding input, and rerank queries and documents
• Output messages, finish reasons, ranked results, errors, and time to first chunk
• Input, output, total, cache-read, cache-write, and reasoning token counts
• Request cost, from LiteLLM’s own cost calculation
• Router calls as parent spans, with each deployment attempt as a child span, when you use wrap_router()

The integration never records embedding vectors.

Install

pip install telemetry-dev-litellm

The package also installs telemetry-dev and litellm.

Python 3.14 isn’t supported.

Trace a call

Set your telemetry.dev project API key and the key for the provider LiteLLM calls. Copy the telemetry.dev key from the project setup page; it starts with td_live_.

export TELEMETRY_DEV_API_KEY=td_live_...
export OPENAI_API_KEY=sk-...

Call init() and instrument_litellm(), then use LiteLLM as usual:

import litellm
import telemetry_dev
from telemetry_dev_litellm import instrument_litellm

telemetry_dev.init(service_name="litellm-app")
instrument_litellm()

try:
    response = litellm.completion(
        model="gpt-4o-mini",
        messages=[
            {"role": "user", "content": "Explain telemetry.dev in one sentence."},
        ],
        max_tokens=200,
    )
    print(response.choices[0].message.content)
finally:
    telemetry_dev.flush()
    telemetry_dev.shutdown()

Run the script, then open the trace explorer. The trace has one chat gpt-4o-mini generation span with the messages, the reply, token usage, and cost.

Without an API key, init() turns telemetry off and LiteLLM works as before.

If you can’t control import order, call the package’s drop-in functions instead. They behave like the LiteLLM functions of the same name and produce exactly one span, with or without global instrumentation:

from telemetry_dev_litellm import completion

response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)

API

These functions take no options.

Traced calls

The integration doesn’t trace text_completion(), image, audio, batch, or video functions such as video_generation(), or provider-specific APIs.

LiteLLM’s own retries from num_retries happen inside one call, so they appear as one span that covers every attempt.

Router calls

A LiteLLM Router dispatches each deployment attempt through the module functions. With instrument_litellm() active, each attempt gets a span. Router retries and fallbacks are separate attempts, so they get separate spans.

To group the attempts, wrap the Router. The Router span becomes the parent of each attempt’s span:

import litellm
from telemetry_dev_litellm import instrument_litellm, wrap_router

instrument_litellm()
router = wrap_router(litellm.Router(model_list=[...]))
router.completion(model="my-group", messages=[...])

wrap_router() instruments the Router’s completion, Responses, embedding, and rerank methods, synchronous and asynchronous.

Streaming

stream=True calls return a wrapper around LiteLLM’s stream. It supports synchronous and asynchronous iteration, context managers, close(), aclose(), and the stream’s own attributes.

A stream span stays open until your code reads the stream to the end, closes it, or the stream fails. When it ends, the integration rebuilds the response with litellm.stream_chunk_builder(). The span records the time to the first chunk.

The integration doesn’t add stream_options, so streamed usage is whatever LiteLLM reports: the provider’s counts when the provider sends them, and otherwise LiteLLM’s estimate. If a stream produced output but LiteLLM reports zero input and zero output tokens, the span records no usage or cost instead of zeros. To get the provider’s own counts, pass stream_options={"include_usage": True} to providers that support it.

Cost

The span records gen_ai.usage.cost from LiteLLM’s response_cost. If the response doesn’t have one, the integration calls litellm.completion_cost(). If LiteLLM can’t price the model, the span has no cost.

Provider names

The integration maps LiteLLM provider names to OpenTelemetry provider names:

Any other provider keeps its LiteLLM name, such as ollama, openrouter, together_ai, or fireworks_ai.

Captured content

Prompts and responses follow the core SDK’s capture_input, capture_output, and mask settings. With capture off, models, finish reasons, usage, cost, timing, and errors are still recorded.

Metadata is stored outside those settings. The integration exports each key of a call’s litellm_metadata argument, or of metadata when litellm_metadata is absent or empty, as td.metadata.<key>. The capture settings don’t remove these values and the mask doesn’t see them, so don’t put credentials or personal data in metadata. For user and session IDs, use propagate_attributes(user_id=..., session_id=...) from the Python SDK instead.

See privacy for server-side redaction.

Send telemetry before exit

In a short script, call telemetry_dev.flush() and then telemetry_dev.shutdown() before the process exits, as in the example above. In serverless code, call flush() before the handler returns. See the Python SDK guide for details.