Bedrock Streaming Token Usage: Handle Incomplete Streams

Your dashboard says a user made 4,212 streamed Bedrock calls last week. Your AWS bill implies a few more tokens than your logs can account for. Nobody has done anything wrong, and nothing is broken. The gap comes from how ConverseStream reports usage: once, at the very end of the stream.

If the stream finishes cleanly, you get input tokens, output tokens and total tokens in one tidy event. If the user closes the tab, your proxy times out, or the connection drops at second eight of a twelve second response, that event never shows up. The text you already forwarded to the user was real. The tokens behind it were real. Your record of them is empty.

This post walks through the ConverseStream event sequence, shows what an interrupted stream actually leaves you with, and argues for a simple discipline: record observed usage and estimated usage as two different things, and never let one pose as the other. Examples are in TypeScript and Python, since that is what most small teams calling Bedrock directly are writing.

Where usage appears in a ConverseStream response

Bedrock streaming token usage lives in exactly one place: the metadata event at the end of a ConverseStream response. That event carries a usage object with inputTokens, outputTokens and totalTokens, plus a metrics object with latencyMs. Depending on the model and features you use, it can include extra fields such as cache read and write token counts. Check the current Converse API reference in the AWS docs for the exact shape, because it has grown over time.

Look at what's missing. No earlier event carries a running token count. The contentBlockDelta events hold text fragments, tool-use input fragments or reasoning content, and nothing else. The messageStart event gives you the role. The messageStop event gives you the stop reason. Neither gives you counts.

That design makes sense for the service. The model has to finish before a final tally means anything. But many tracking implementations quietly ignore the consequence: usage belongs to a completed stream, not a started one.

If you have written code like usage = event.get('metadata', {}).get('usage') inside your event loop, you have already written the happy path. Everything that follows in this post is about the other path.

My stance: treat the metadata event as a receipt, not a meter. A meter ticks while the service is running. A receipt prints when the transaction closes. If the transaction never closes cleanly on your side, you don't have a receipt, and you shouldn't pretend you do.

The event order: messageStart, content blocks, messageStop, metadata

A normal ConverseStream response arrives in a fixed order. Knowing it precisely lets you tell, in code, how far a stream got before it ended.

1. messageStart: one event, carrying role (assistant).

2. contentBlockStart: only for blocks that need a start marker, such as tool use. Plain text blocks often skip it.

3. contentBlockDelta: many events, each with a delta (text, tool input JSON fragment, or reasoning content) and a contentBlockIndex.

4. contentBlockStop: one per block.

5. messageStop: one event with stopReason, such as end_turn, max_tokens or tool_use.

6. metadata: one event with usage, metrics and optionally trace information.

The detail that bites: messageStop comes before metadata. Teams often treat messageStop as the end of the stream and close the response, record the span, or flush logs right there. Do that and you throw away the one event that holds your token counts. Keep reading until the stream is exhausted.

A minimal Python loop that respects this looks like:

usage = None
stop_reason = None
for event in response["stream"]:
    if "contentBlockDelta" in event:
        handle_delta(event["contentBlockDelta"])
    elif "messageStop" in event:
        stop_reason = event["messageStop"]["stopReason"]
    elif "metadata" in event:
        usage = event["metadata"]["usage"]

In TypeScript with the AWS SDK v3, the same shape applies when you iterate response.stream from ConverseStreamCommand: check which key is present on each event and capture metadata.usage when it appears.

After the loop, usage is either populated or it isn't. That binary is the whole problem, and your data model should say so out loud.

What an interrupted stream leaves you with

Walk through the failure cases and ask the same question each time: what do I hold in memory right now?

Client cancellation. The user closes the tab or hits stop. You abort the upstream call. You hold some number of contentBlockDelta fragments and no messageStop, no metadata.

Timeout. Your server or gateway enforces a deadline, say 30 seconds on a serverless function. The loop is cut off mid-stream. Same state: partial text, no counts.

Dropped connection. A network error raises an exception inside the stream iterator. Some SDK versions throw a modeled stream exception such as modelStreamErrorException or internalServerException as an event inside the stream. Others surface a transport error. Either way you get partial content and no usage.

Throttling mid-stream. A throttlingException can arrive after content has started flowing. Again, no final metadata.

In all four cases the model may have generated and processed a substantial number of tokens. The input was fully processed before the first delta arrived. Whether you're billed for the interrupted call is a question for your AWS billing data and the Bedrock pricing documentation. You can't derive it from your own logs, so don't assume the answer in either direction.

Here is the claim I'd hold firmly: you cannot recover exact counts after the fact from what you have. You can tokenize the text fragments locally to get an approximation, but a local tokenizer is not the model's tokenizer. Input tokens (system prompt, history, tool definitions, any images or documents) don't appear in the output stream at all.

So you have two honest options. Record nothing and accept the gap, or record an estimate and label it as one. Pretending the metadata event arrived is the only wrong choice.

Observed usage vs estimates: label them differently

The fix is a data modeling decision, not a clever algorithm. Store usage with a provenance field, and never merge the two kinds into one column without it.

A workable schema for each model call:

• usage_source: observed, estimated or unknown

• input_tokens, output_tokens: nullable integers

• stream_completed: boolean (did you see messageStop and metadata?)

• stop_reason: string or null

• estimate_method: string or null, such as chars_div_4 or local_tokenizer_v1

When the metadata event arrives, write usage_source = observed with the numbers Bedrock gave you. When the stream ends early, you pick one of two behaviors.

The conservative one: set both counts to null, usage_source = unknown, and stream_completed = false. Your cost reports then show a count of calls with unknown usage next to the summed cost, which is exactly the right amount of discomfort.

The pragmatic one: compute an estimate from the text you received, write it with usage_source = estimated, and record how you made it. You can approximate output tokens from received text. You can approximate input tokens from the request you built, since you know what you sent. Both will drift from reality. That's fine as long as the label travels with the number.

The rule that matters: dashboards and alerts must be able to filter on provenance. A weekly cost chart that sums observed and estimated tokens into one line hides exactly what you need when AWS billing disagrees with your logs.

OpenTelemetry's GenAI semantic conventions define attributes for input and output token usage on model spans. They don't say whether a value was measured or guessed. A custom attribute for provenance, set by your own code, fills that gap.

Handling cancellations, timeouts and mid-stream errors in your code

Every exit path from the stream loop must write a usage record. The common bug is a return or throw that skips the recording step.

In Python, wrap the loop in try/finally so the record is always written, and decide provenance from local state:

usage, stop_reason, text_parts = None, None, []
completed = False
try:
    for event in response["stream"]:
        if "contentBlockDelta" in event:
            d = event["contentBlockDelta"]["delta"]
            if "text" in d:
                text_parts.append(d["text"])
        elif "messageStop" in event:
            stop_reason = event["messageStop"]["stopReason"]
        elif "metadata" in event:
            usage = event["metadata"]["usage"]
            completed = True
finally:
    record_call(
        usage=usage,
        completed=completed,
        stop_reason=stop_reason,
        partial_text="".join(text_parts),
    )

In TypeScript, the equivalent is a try/finally around a for await loop. Watch cancellation closely: if you pass an AbortSignal to client.send(command, { abortSignal }), an abort throws inside the loop, and your finally block is the only place guaranteed to run. Keep it cheap and non-throwing, because it runs while the request is already failing.

Three practical rules:

1. Record on the way out, not on the way in. A span or log row created at stream start and updated at the end is fine, but the update must live in finally.

2. Distinguish who ended the stream. Cancelled by the user, timed out by you, errored by the service, and completed normally are four different outcomes. Store the reason. Cancellation rates and error rates are different signals.

3. Keep the loop reading after messageStop. Break only when the iterator is exhausted, or you will lose metadata even on healthy streams.

If you run behind a proxy or a serverless platform, a client disconnect may not propagate upstream promptly. Your function can keep consuming a stream nobody is reading. Propagate the abort explicitly.

Tracing Bedrock streams with telemetry.dev: what is reported and what is not

telemetry.dev has provider instrumentation for AWS Bedrock in its first-party TypeScript and Python SDKs, and it works with any OTLP/HTTP exporter or OpenTelemetry Collector. It traces model calls and tracks tokens, cost, latency and errors, so a streamed Bedrock call shows up as a span you can inspect alongside the tool steps and retrievals around it. If you already trace with the Vercel AI SDK, the setup is similar to what is described in How to Trace Vercel AI SDK Calls with OpenTelemetry.

The boundary follows directly from how Bedrock reports usage, so it's worth stating plainly. Computed cost in telemetry.dev depends on reported token usage and a matching model price. For a stream that completes, the usage the model reports in the final metadata event feeds that calculation. For a stream that never delivers that event, there is no reported usage to compute from, so don't expect an exact token count or cost for that call. A span can still tell you the call happened, how long it ran, and that it ended in an error or was cut short. That helps: a silent gap in your numbers becomes a visible failed or interrupted call you can click into.

If you want an estimate on an interrupted call, you can record one yourself. If you do, apply the provenance rule from earlier and mark it as an estimate in your own attributes so it never reads as a measurement.

Two more notes. Prompt capture can be turned off per environment before storage, which matters if your Bedrock requests carry user content. Redaction patterns apply, but they can't catch all sensitive data, so treat capture settings as the primary control. The free plan covers 30k units, which is enough to trace a development workload and see how your own interrupted streams look.

A checklist for streaming usage you can trust

Run this against your current ConverseStream code. Each item closes a specific gap.

Reading the stream

• Does your loop continue past messageStop until the iterator is exhausted?

• Do you capture metadata.usage (and cache token fields if you use prompt caching)?

• Do you handle in-stream exception events, not just transport errors?

Recording the result

• Is the usage record written in a finally block or equivalent, on every exit path?

• Does each record carry usage_source (observed, estimated, unknown)?

• Do you store stream_completed, stop_reason and who ended the stream (user, timeout, service error)?

• Are null counts allowed in your schema, so unknown stays unknown?

Estimating, if you choose to

• Is every estimate labeled with its method?

• Can a dashboard filter observed from estimated?

• Do you report the number of calls with unknown or estimated usage next to total cost?

Reconciling

• Do you compare your observed totals with AWS billing data periodically, and expect a gap that tracks your interruption rate?

• Do you alert when the share of streams without a metadata event rises? A jump usually means a deploy changed timeouts or a client started cancelling more.

Cancellation hygiene

• Does a client disconnect abort the upstream Bedrock call, or does your server keep consuming it?

• Are abort paths tested, not just the success path?

If you can answer yes to most of these, your streaming numbers are honest even where they're incomplete. That beats a clean-looking total that quietly assumes every stream finished.

Conclusion

Interrupted streams are not an edge case you can wave away. They are a measurable share of real traffic, and each one is a call whose token count Bedrock never handed you. Record what you observed, label what you estimated, and keep the unknowns visible.

To see where your own streams end early, add the Bedrock instrumentation from telemetry.dev's TypeScript or Python SDK to one service on the free plan, then open the spans for calls that errored or ran long. Compare those against the calls that reported usage. The difference between the two groups is the gap in your cost data, and now you can see it.