Amazon Bedrock

Trace Amazon Bedrock Runtime and Agent Runtime calls from TypeScript or Python.

Wrap your Bedrock Runtime or Agent Runtime client, and each supported call you make with it becomes a span in telemetry.dev. The integration never changes request objects, so AWS SigV4 signatures stay valid.

What you get

• Converse messages, system instructions, inference parameters, guardrail data, native InvokeModel bodies, and agent input
• Model output, agent output, citations, return-control data, the AWS request ID, HTTP status, retry counts, and errors
• Token usage from responses, native model bodies, AWS headers, and agent trace events
• Spans for embedding, guardrail, knowledge-base retrieval, rerank, and flow calls, and time to first chunk for streams

Images, documents, video, and audio sent inline appear in span input as parts without their bytes. InvokeModel bodies that aren’t JSON aren’t recorded.

Install

Install the telemetry packages and the Bedrock Runtime client:

npm i @telemetry-dev/sdk @telemetry-dev/bedrock @aws-sdk/client-bedrock-runtime

pip install telemetry-dev-bedrock

The Python package also installs telemetry-dev and boto3. For Agent Runtime in TypeScript, also install the Agent Runtime client:

npm i @aws-sdk/client-bedrock-agent-runtime

Trace a call

Set your telemetry.dev project API key, and make sure your AWS credentials are available to the AWS SDK. Copy the telemetry.dev key from the project setup page; it starts with td_live_.

export TELEMETRY_DEV_API_KEY=td_live_...

Call init(), wrap the client, and use it as usual:

import {
  BedrockRuntimeClient,
  ConverseCommand,
} from "@aws-sdk/client-bedrock-runtime";
import { flush, init, shutdown } from "@telemetry-dev/sdk";
import { wrapBedrock } from "@telemetry-dev/bedrock";

init({ serviceName: "bedrock-app" });
const client = wrapBedrock(
  new BedrockRuntimeClient({ region: "us-east-1" }),
);

try {
  const response = await client.send(
    new ConverseCommand({
      modelId: "anthropic.claude-3-5-haiku-20241022-v1:0",
      system: [{ text: "You are concise." }],
      messages: [
        { role: "user", content: [{ text: "Say hello from Bedrock." }] },
      ],
      inferenceConfig: { maxTokens: 128, temperature: 0.2 },
    }),
  );
  console.log(
    response.output?.message?.content?.find((part) => "text" in part)?.text,
  );
} finally {
  await flush();
  await shutdown();
}

import boto3
import telemetry_dev
from telemetry_dev_bedrock import wrap_bedrock

telemetry_dev.init(service_name="bedrock-app")
bedrock = wrap_bedrock(
    boto3.client("bedrock-runtime", region_name="us-east-1"),
)

try:
    response = bedrock.converse(
        modelId="anthropic.claude-3-5-haiku-20241022-v1:0",
        system=[{"text": "You are concise."}],
        messages=[
            {"role": "user", "content": [{"text": "Say hello from Bedrock."}]},
        ],
        inferenceConfig={"maxTokens": 128, "temperature": 0.2},
    )
    print(response["output"]["message"]["content"][0]["text"])
finally:
    telemetry_dev.flush()
    telemetry_dev.shutdown()

Run the script, then open the trace explorer. The trace has one chat anthropic.claude-3-5-haiku-20241022-v1:0 generation span with the messages, the reply, and token usage.

Without an API key, init() turns telemetry off and the wrapped client works as before.

Trace Agent Runtime calls

In TypeScript, import the Agent Runtime functions from @telemetry-dev/bedrock/agents. In Python, wrap_bedrock() also accepts a bedrock-agent-runtime client.

Pass enableTrace: true (enableTrace=True in Python) to InvokeAgent so AWS sends the trace events that carry the agent’s token usage.

import {
  BedrockAgentRuntimeClient,
  InvokeAgentCommand,
} from "@aws-sdk/client-bedrock-agent-runtime";
import { wrapBedrockAgents } from "@telemetry-dev/bedrock/agents";

const agent = wrapBedrockAgents(
  new BedrockAgentRuntimeClient({ region: "us-east-1" }),
);

const response = await agent.send(
  new InvokeAgentCommand({
    agentId: process.env.BEDROCK_AGENT_ID!,
    agentAliasId: process.env.BEDROCK_AGENT_ALIAS_ID!,
    sessionId: crypto.randomUUID(),
    inputText: "Say hello",
    enableTrace: true,
  }),
);

for await (const event of response.completion ?? []) {
  if (event.chunk?.bytes) {
    process.stdout.write(new TextDecoder().decode(event.chunk.bytes));
  }
}

import os
import uuid

import boto3
from telemetry_dev_bedrock import wrap_bedrock

agent = wrap_bedrock(
    boto3.client("bedrock-agent-runtime", region_name="us-east-1"),
)

response = agent.invoke_agent(
    agentId=os.environ["BEDROCK_AGENT_ID"],
    agentAliasId=os.environ["BEDROCK_AGENT_ALIAS_ID"],
    sessionId=str(uuid.uuid4()),
    inputText="Say hello",
    enableTrace=True,
)

for event in response["completion"]:
    if "chunk" in event:
        print(event["chunk"]["bytes"].decode(), end="")

The agent span finishes when you read the completion stream to the end. Call init() and flush as in the first example.

Instrument all clients

To trace every client without wrapping each one, call the global function once at startup:

import { instrumentBedrock } from "@telemetry-dev/bedrock";
import { instrumentBedrockAgents } from "@telemetry-dev/bedrock/agents";

instrumentBedrock();
instrumentBedrockAgents();

from telemetry_dev_bedrock import instrument_bedrock

instrument_bedrock()

In TypeScript, global instrumentation patches send() on the BedrockRuntimeClient and BedrockAgentRuntimeClient prototypes, so other AWS clients aren’t affected. Use either global instrumentation or wrapBedrock() for a client, not both.

In Python, global instrumentation wraps Botocore’s API call method and acts only on bedrock-runtime and bedrock-agent-runtime clients. It skips clients you wrapped with wrap_bedrock(), so they don’t get a second span.

API

boto3 has no asynchronous client, and the Python package doesn’t instrument aiobotocore.

Options

Traced calls

In TypeScript these are the ...Command classes, such as ConverseCommand. In Python they are the snake_case client methods, such as converse() and invoke_agent(). For RetrieveAndGenerate and Rerank, {model} is the last segment of the model ARN.

The TypeScript integration recognizes a command by its class name. If your bundler minifies class names, commands pass through without a span; keep class names in the build (for example, with esbuild’s keepNames).

A rerank span records the queries, inline sources, model request fields, ranked results and scores, result counts, and pagination metadata.

InvokeModel token usage depends on the model. Anthropic Claude, Amazon Titan and Nova, Meta Llama, and Titan embedding models return known token fields. Cohere and Mistral text models don’t always include usage in the response body, but their spans still record the bodies and finish reasons.

Other operations pass through without a span, including CountTokens, InvokeModelWithBidirectionalStream, the async invoke operations, Agent Runtime session management, GenerateQuery, and OptimizePrompt.

Streaming

A stream span stays open until your code reads the stream to the end, stops early, or the stream fails. The wrapper reads events only as you consume them, so backpressure is kept. On an early stop or an error, the span records the partial output and is marked as incomplete with telemetry.dev.capture.truncated.

In TypeScript, a stream that is garbage-collected before it finishes also ends its span, on runtimes that support FinalizationRegistry.

In Python, the integration reads an InvokeModel response’s StreamingBody once, then replaces it with a new body over the same bytes, so response["body"].read() still works. Event streams pass through lazily. Closing one early, or a GeneratorExit, ends the span with the partial output.

Provider names

Every span uses the provider amazon-bedrock. The model is the modelId, modelArn, or foundationModel you sent.

Calls through Anthropic’s AnthropicBedrock client are traced by the Anthropic integration and use the provider aws.bedrock instead.

Captured content

Prompts and responses follow the core SDK’s captureInput, captureOutput, and mask settings (capture_input, capture_output, and mask in Python). With output capture off, request IDs, models, finish reasons, usage, timing, and errors are still recorded. AWS error messages are recorded on the span whatever the capture settings. For agent trace events, see the warning under Options.

Send telemetry before exit

In a short script, call flush() and then shutdown() before the process exits, as in the example above.

In serverless code, initialize with exportMode: "immediate" (export_mode="immediate" in Python) so each span is sent when it ends, and still call flush() before the handler returns: immediate mode doesn’t send metrics right away. See the TypeScript SDK and Python SDK guides for details.