Pi coding agent

Trace pi sessions, prompts, model calls, tool executions, usage, cost, and lifecycle events.

@telemetry-dev/pi is a pi package for the pi coding agent (@earendil-works/pi-coding-agent). Install it, set an API key, and each pi session appears in telemetry.dev as a trace with its prompts, model calls, tool calls, token usage, and cost.

By default the package sends a lot of content: the full request pi sends to the model provider on every model call (system prompt, conversation so far, and tool results), the model’s complete responses including thinking, and every tool call’s arguments and results. You can turn content off with two options; see What leaves your machine.

Before you start

• pi 0.80.4 or later. Version 0.1.9 of the package is built and tested against pi 0.82.1.
• A telemetry.dev project and its API key. Keys start with td_live_.

Set up

export TELEMETRY_DEV_API_KEY=td_live_...
export TELEMETRY_DEV_ENVIRONMENT=development

pi install npm:@telemetry-dev/pi

Load the extension from a file instead

Use an extension file when you want to set options in code, such as turning off content capture or adding a mask. Don’t also keep the package in settings.json: pi would load the extension twice and record every span twice.

Create ~/.pi/agent/extensions/telemetry-dev.ts:

import { telemetryDevExtension } from "@telemetry-dev/pi";

export default telemetryDevExtension({
  captureInput: false,
  captureOutput: false,
});

The file must be able to import @telemetry-dev/pi, so install the package in a node_modules directory that Node.js module resolution finds from the file’s location, such as ~/.pi/agent.

Check that data arrives

Send a prompt and wait for pi to finish it. Then open the trace explorer. The extension starts sending a prompt’s spans when the agent loop ends, and sends the rest when the prompt settles.

If no trace appears, check that TELEMETRY_DEV_API_KEY is set in the shell that started pi. Without a key the extension does nothing and reports no error.

What is recorded

All prompts in one pi session share one trace:

session
└── invoke_agent
    ├── chat {model}
    │   ├── execute_tool {toolName}
    │   └── execute_tool {toolName}
    └── chat {model}

• session covers the whole pi session. Each prompt adds an invoke_agent child.
• invoke_agent records the prompt, including image blocks, the final assistant content, the result, and the session ID.
• chat {model} records the provider, models, response ID, finish reason, token usage, and pi’s cost estimate. It also records the request pi sent to the provider, the system prompt, and every content block the model returned.
• execute_tool {toolName} records the tool call ID, arguments, result, and error. It sits under the chat span that issued the call, or under invoke_agent when no message matches.
• Session, turn, compaction, model-change, and shutdown events become logs.

The cost is pi’s own estimate from its bundled model prices, sent as gen_ai.usage.cost. It isn’t a charge from your provider.

Every span and log reads the current pi session ID and records it as gen_ai.conversation.id, so /new, /resume, and /fork keep each session’s telemetry separate.

How model requests are captured

The extension captures each model request at pi’s before_provider_request event, after pi’s context handlers and its conversion to the provider’s message format. That conversion removes !! shell output and custom-message details. The system prompt is the effective one pi reports at that point.

The captured request isn’t guaranteed to be the exact payload on the wire, because a later before_provider_request handler can replace it. Custom providers must call pi’s onPayload callback for the request and system prompt to be captured; without it, those two fields are missing, and prompts, responses, and tool calls are still captured.

What leaves your machine

By default the extension sends:

• Your prompts, including attached images (captureInput).
• The full provider request on every chat span, which contains the system prompt and the conversation so far, including earlier tool results (captureInput).
• Tool arguments (captureInput), such as shell commands and file paths.
• Every content block the model returns: text, thinking, tool calls, and provider signatures (captureOutput).
• Tool results (captureOutput), such as file contents and command output.

Because each chat span carries the conversation so far, the same content is sent again on every model call in a session.

Set captureInput and captureOutput to false in an extension file to stop sending this content. mask can change or drop individual values before they are sent. The package installed with pi install reads only environment variables, so it always uses the defaults.

These are sent whatever the options:

• Session IDs, the working directory (on the session start log), provider and model names, token counts, cost, and lifecycle metadata.
• The result text of a failed tool, as the span’s error message. In version 0.1.9 this ignores captureOutput and mask.
• Provider error messages on failed model calls.

telemetry.dev’s privacy settings can also drop recognized prompt and response content before storage, but only after it has left your machine.

How it runs

agent_start opens the invoke_agent span. Retries and compaction can start the agent loop again, but they stay in the same span.

agent_end records the current result and starts an export. agent_settled closes invoke_agent and any unfinished tool spans, then exports again. This order lets a retry replace a retryable result before the prompt settles.

session_shutdown closes an unfinished prompt and the session span, then waits for the last export. Switching sessions doesn’t shut the extension down.

Pi doesn’t report time to first token or request duration. The extension measures each chat span from message_start to message_end with the local clock and records no time to first token.

The extension uses pi’s tool_execution_start and tool_execution_end events, not tool_call and tool_result. In pi, an error in a tool_call handler blocks the tool, and telemetry should never do that.

Configuration

In an extension file, telemetryDevExtension() accepts these options, which override the environment variables:

The extension also accepts the other TypeScript SDK init() options, except registerGlobal, which it always sets to false. The first extension instance to start in a process sets the options for that process.

Limitations

• An assistant message with stopReason: "aborted" gets the finish reason aborted, not an error. Only stopReason: "error" marks spans as errors.