TanStack AI

Trace TanStack AI chat, embedding, summary, and media calls with native @telemetry-dev/tanstack-ai middleware.

Add the telemetryDev() middleware to a TanStack AI chat() call and each call becomes an OpenTelemetry trace in telemetry.dev, with a span per model iteration and tool call, plus token usage and errors. The same middleware also traces embeddings, summaries, and image, speech, transcription, and video generation.

@telemetry-dev/tanstack-ai is a native TanStack AI middleware. It exports through its own tracer provider and does not change your global OpenTelemetry setup.

Install

npm install @telemetry-dev/tanstack-ai @tanstack/ai

pnpm add @telemetry-dev/tanstack-ai @tanstack/ai

yarn add @telemetry-dev/tanstack-ai @tanstack/ai

bun add @telemetry-dev/tanstack-ai @tanstack/ai

chat() tracing supports @tanstack/ai 0.28.0 and later 0.x releases. decide() requires @tanstack/ai 0.57.0 or later. Tracing of the other activities is tested against @tanstack/ai 0.57.0.

Set the API key

Create a project at telemetry.dev and copy an API key from the project setup page. Keys look like td_live_....

export TELEMETRY_DEV_API_KEY=td_live_...

Without an API key, telemetryDev() returns a named middleware that does nothing. Your calls run as usual.

Trace a chat call

Add telemetryDev() to the middleware array of the chat() call:

import { chat } from "@tanstack/ai";
import { telemetryDev } from "@telemetry-dev/tanstack-ai";

const stream = chat({
  adapter,
  messages,
  metadata: { userId: "u_123", sessionId: "s_456", tenant: "acme" },
  middleware: [telemetryDev()],
});

The middleware records chat metadata like this:

Without metadata.sessionId, the middleware uses the TanStack AI threadId as the conversation ID.

Share one middleware instance

You can create one instance at module scope and use it in every handler:

const telemetry = telemetryDev();

chat({ adapter, messages, middleware: [telemetry] });

One instance is safe for overlapping calls. The middleware keeps separate state for each call’s middleware context, so one call cannot add spans or usage to another.

Check the trace

Run a chat call, then open Traces in your project. Each chat() call shows a root span (operation chat, or invoke_agent once the call uses tools), one chat span per model iteration, and an execute_tool span per tool call.

Calls that share a conversation ID share one trace. Each call’s root span appears as a sibling in start order.

Trace other activities

Pass the same middleware to embeddings, summaries, and media generation:

const telemetry = telemetryDev();

await embed({ adapter, input: "hello", middleware: [telemetry] });
await generateImage({ adapter, prompt, middleware: [telemetry] });

Each call makes one span:

These spans record the provider, requested model, request ID, the model the result reports (except for embed()), duration, errors, aborts, and the token usage and cost the provider reports. They do not record prompts, input text, summaries, transcripts, decision questions or answers, or image, audio, and video payloads. Error messages and abort reasons are recorded as-is and can contain sensitive text.

Video jobs

TanStack AI calls middleware at two points in a video job:

• generateVideo() without stream makes a span when the provider accepts the job. TanStack AI calls the middleware only after it creates the job, so this span marks acceptance and has no meaningful duration.
• getVideoJobStatus() makes a span only for the poll that sees the job completed or failed, with the usage the provider reports. Polls that see pending or processing make no span.

Both spans carry the job ID as gen_ai.response.id. When no span is active, they share one trace derived from the job ID, so a job created in one process and finished in another still lands in one trace.

generateVideo({ stream: true }) makes one span from job creation to the final state.

What is traced

For chat() calls:

Fields the adapter does not report are left out, not set to zero.

What is not traced

• Time to first chunk. TanStack AI gives middleware no first-chunk signal.
• rerank(), generateAudio(), generateVoice(), generateWorld(), and generateLiveVideo(). They accept the middleware but produce no spans or metrics.
• Conversation links for activities other than chat(). metadata, userId, and sessionId apply to chat() only.
• Billing units other than tokens, such as seconds, characters, or images.

Known gaps in TanStack AI 0.57.0

• A streamed video run that fails carries no job ID, because TanStack AI does not pass it to middleware before completion.
• A summarize({ stream: true }) that fails mid-stream with a RUN_ERROR chunk reaches the middleware’s onFinish, not onError, so its span records success.
• When an embed(), generateSpeech(), or generateTranscription() adapter rejects with null or undefined, TanStack AI skips the middleware error hook, so the span for that call never ends.

Options

Pass options to telemetryDev(). Explicit options take precedence over environment variables, which take precedence over defaults.

The middleware reports instrumentation and export errors to onError. These errors, including errors thrown by your onError callback, never reach your TanStack AI call.

Serverless runtimes

By default, the final middleware hook waits for the export to finish. To return sooner, pass the export promise to your platform’s lifetime function:

const telemetry = telemetryDev({
  waitUntil: (promise) => ctx.waitUntil(promise),
});

On runtimes without process.env, pass apiKey directly. The middleware sends data with fetch.

Troubleshooting

If no trace appears:

• Check that TELEMETRY_DEV_API_KEY is set in the process that runs the call, or pass apiKey.
• Read the chat() stream to the end. The trace is sent when the run finishes, errors, or aborts.
• Pass onError: console.error to see export errors, such as a rejected API key.