The Cache-Read Tax

The data.

How it was measured.

Every figure is one run of the repository's own measurement tool over its local agent session transcripts, read on the date shown. Nothing is estimated and nothing is carried forward from an earlier measurement.

Cost weighting is applied because the raw counts answer a different question than the one that matters. Generated output is weighted 5x, cache writes 1.25x and cache reads 0.1x against base input, reflecting how those token classes are actually priced.

The headline is the gap between two correct measurements of the same data. By raw count, generated output is 0.27% of tokens, which reads as noise and suggests that shortening what an agent writes is pointless. Cost-weighted, it is 10.6%, which is not noise. Both numbers are true; only one of them describes the bill.

This organization's own always-loaded engineering guidance carried the raw-count framing and drew the conclusion that optimizing output addresses a third of a percent of the problem. That conclusion does not survive weighting, and the measurement tool now says so in its own output.

Context re-read dominates under both measurements, which is the finding that holds either way: an agent turn pays for the whole conversation so far, every time, so cost scales with context size multiplied by turn count rather than with anything the agent produces.

What this does not show.

• One organization's sessions on one tool. Session shapes differ enormously between teams, and a team that clears context aggressively would produce a very different distribution.
• A reading on the date shown. The tool also grades this corpus against thresholds this project set for itself, and a grade is a comparison against thresholds this project set for itself, not an industry benchmark.
• The weighting factors are the ones this project uses. They approximate a real price list but they are not a quotation, and a different weighting moves the 10.6% figure.
• Cost-weighted share is not the same as opportunity. Output being 10.6% of spend does not establish that shortening output is the most effective available saving; it only refutes the claim that it is negligible.
• No causal claim is made. This measures where tokens went, not which practices caused it.

Revisions.

This URL is permanent. When the data is refreshed the version bumps and a row lands here, so a citation made today still resolves to the finding it cited.

Back to all research