Direct answer
Caching saves money only when repeated reads repay the write
Compare the ordinary-input cost of sending the same prefix on every request with one cache write followed by discounted cache reads. The interactive worksheet also models misses, reports the first profitable reuse count, and keeps every rate editable so it never has to pretend one provider's pricing applies to another.
Model a repeated prompt prefix
Paste the current rates for the exact model and provider you use. The starter values are an illustrative unit-rate worksheet—not current pricing for any model.
This scenario saves on the repeated prefix
The comparison covers cacheable input-prefix tokens only. Identical output and non-cacheable input costs cancel from both sides.
Expected savings
$1.51
75.7% versus sending the prefix as ordinary input on every request.
Without caching
$2.00
2,000,000 prefix tokens billed at the normal input rate
Hit-rate scenario
$0.486
1 write + 17.1 expected hits + 1.9 ordinary-input misses
Every-reuse-hits ceiling
$0.315
$1.69 maximum modeled savings if all 19 reuses hit the same cache
Break-even reuses
1
Minimum successful reads after one write for a positive full-hit saving; TTL misses can move this higher.
What this hit-rate scenario assumes
There is one initial cache write. Each later attempt is either a cache read or an ordinary-input miss. Some providers automatically rebuild an expired cache or charge storage, throughput, or TTL-related fees; add those separately when their billing rules require it. An observed cache share is not a savings percentage because the write, read, and normal-input rates still determine the money saved.
Cost path
How prompt-caching savings work
The useful comparison is not “cached tokens versus total tokens.” It is the cost of one stable prefix across the same set of requests. Separate the first write, successful reads, and misses so each event receives the correct rate.
- 01 · WRITE
Create an eligible prefix
Keep system instructions, tool schemas, examples, or reference material stable and early in the request. Minimum size and matching rules vary by provider.
- 02 · REUSE
Send requests before expiry
Each compatible request can read the cached prefix. A TTL, changed leading token, model switch, or routing decision can turn an expected hit into a miss.
- 03 · VERIFY
Measure billed cache events
Use provider usage fields and invoices—not prompt similarity alone—to confirm reads, writes, misses, retention charges, and the rates actually applied.
Worked example
A 100k-token prefix reused across 20 requests
These normalized worksheet rates demonstrate the math; they are not current pricing for a provider or model.
- Cacheable prefix
- 100,000 tokens
- Requests
- 20 total
- Normal input
- $1.00/M
- Cache write
- $1.25/M
- Cache read
- $0.10/M
- Expected hit rate
- 90% of reuses
No cache
$2.00
90% hit scenario
$0.486
Expected savings
$1.514
The 19 reuse attempts become 17.1 expected hits and 1.9 expected misses. One write costs $0.125, expected reads cost $0.171, and misses cost $0.19. The fractional events are an expected-value planning model across many comparable batches—not a claim that one batch can contain a fraction of a request.
Good candidates
Cache stable, expensive prefixes
- Large shared instructions. Long system prompts and few-shot examples reused across a burst of requests can amortize a write quickly.
- Repeated reference context. A document, policy, or code map read by many compatible requests may create a substantial cacheable share.
- Stable tool definitions. Agent tool schemas often repeat, but confirm that the provider includes them in its cache matching and usage fields.
Common traps
Model expiry and misses explicitly
- TTL shorter than the workload. Multiple cache lifetimes can mean multiple writes; calculate each lifetime as a separate batch.
- Dynamic content too early. Put user-specific or rapidly changing content after the shared prefix when the provider's matching rules allow it.
- Savings inferred from cache share. Token share is useful telemetry, but the rate spread and write cost determine currency savings.
Reference
Frequently asked questions
What is prompt caching?
Prompt caching lets an eligible stable prefix—such as system instructions, tool definitions, or a large reference document—be reused without processing and billing every repeated token as ordinary input. The exact matching rules, minimum prefix size, lifetime, and rates are provider-specific.
How many reuses does a prompt cache need to break even?
It depends on the write premium and the difference between normal-input and cache-read rates. This calculator finds the first profitable successful reuse count after one write. Expiration, misses, or another write can move the real break-even point higher.
Does a 90% cache hit rate mean 90% cost savings?
No. A hit rate describes how often eligible reuse attempts hit. Savings also depend on the share of each request that is cacheable, the write charge, the read discount, misses, and unchanged output or non-cacheable input costs.
Why does changing a prompt prefix reduce cache hits?
Caching generally relies on an identical or provider-compatible prefix. Put stable instructions and examples first, then move request-specific text later. Even a small early change may make later content ineligible for reuse under the provider's matching rules.
Does this estimate include output tokens or subscription plans?
No. It isolates the repeated input prefix so the caching decision is visible. Identical output and non-cacheable input costs cancel in the comparison. The estimate is API-equivalent arithmetic, not a subscription allowance or provider invoice.
Sources
- LiteLLM model pricing dataset
Public model pricing reference used for API-equivalent estimates; provider billing remains authoritative.
- OpenAI prompt caching documentation
Official eligibility, cache-hit reporting, retention, and pricing guidance for OpenAI API requests.
- Claude prompt caching documentation
Official cache breakpoint, minimum-token, lifetime, write, and read behavior for Claude models.
- Gemini context caching documentation
Official implicit and explicit context-caching behavior, including model eligibility and explicit-cache storage duration.