What telemetry.dev does

Trace model calls, tool steps, and MCP requests. Get an alert when cost or errors break from normal, change prompts without a deploy, and trace each eval run.

Start tracing free

Features

Tracing

Follow model calls, tool steps, and retrievals in one trace, with the duration of each step.

See tracing
Tracessupport-agent5 spans · 4.9s
Span01s2s3s4s
support-agent4.9s
lookup_order212ms
chat gpt-4.1-mini845ms
track_shipment2.42s
chat gpt-4.1-mini1.16s
Selected
track_shipment
Duration
2,418ms
Share of run
49%

MCP analytics

Spans, failures, and p95 duration by method and tool, from the client and the server.

See MCP analytics
MCPToolLast 7 days
ToolSpansp95Failed
create_refund2,3302.4s18.0%
get_invoice5,912640ms1.1%
check_inventory4,208310ms0.2%
cancel_order1,077880ms0.6%

Alerts

Email alerts for cost, budgets, agent loops, and model or tool errors.

See alerts
Alerts
  • Tool errorsAlerting
  • Monthly budgetAll clear
  • Agent loopAll clear
  • Missing pricesNeeds more data

Prompt management

Versions, labels, and rollouts without a deploy. Compare the versions on real calls.

See prompt management
Metrics
VersionCallsErrorsp50Cost/call
v81,2040.4%1.1s$0.0006
v710,8811.9%1.4s$0.0009
v62,3102.2%1.5s$0.0009

Evals

Trace each eval run, with its cost and errors, as you trace production.

See evals
lookup_ordererror
Error
TimeoutError
The operation was aborted due to timeout
Parent
chat gpt-4.1 · case 2
evaluate refund-policy

Connect your application

Use a telemetry.dev integration, or keep your OpenTelemetry setup and change where it exports.

Trace your next AI request

Choose an integration, add your project API key, and run an instrumented request.

Start free

30,000 ingestion units a month, one per span, log record, or metric data point. No credit card required.

npm install @telemetry-dev/sdk
Start tracing free

Talk to an engineer

Our team shows you tracing, cost tracking, and privacy controls for the frameworks and model providers that you use.

Book a call