Prompt Caching, Explained: How It Works at OpenAI, Anthropic and Google, and What It Saves

Prompt caching lets an AI API reuse the opening part of a request it has already processed and bill that part at a fraction of the normal input price. On current models from OpenAI, Anthropic and Google, a cached token costs 2.5% to 10% of a regular input token.

Our pillar on what AI actually costs names caching as one of the two largest API discounts. This satellite covers how prompt caching works, how each vendor runs it, how to order a prompt so the cache hits, a worked example, and when caching saves nothing.

The short version: put what stays the same at the top of the prompt and what changes at the bottom. OpenAI and Google cache automatically. Anthropic caches when you ask. Anthropic and OpenAI’s newest models charge extra to write the cache, so caching pays off only when enough requests reuse it. Google’s older explicit cache bills hourly storage instead.

How prompt caching works

Every request is input: instructions, tool definitions, reference documents, the conversation so far and the new question. We explain the split in input vs output tokens.

The model processes all of that input before it writes. Prompt caching saves that work for the first part of the prompt, called the prefix. If the next request starts with the same prefix, the vendor skips the work and charges the cached rate.

Three rules apply everywhere.

The match is exact and starts at the beginning. One changed character ends the match there. Everything after it is billed at full price.

The cache expires. Entries live for minutes or hours. Traffic slower than that never hits.

There is a minimum size. Short prompts are not cached, and Anthropic’s docs say no error is returned. Check the usage numbers.

Caching also shortens the wait before a response starts. It cuts input cost only, never output.

Prompt caching at OpenAI, Anthropic and Google, as of October 2026

Checked on each vendor’s docs and pricing pages on October 21, 2026. A write stores the prefix. A read reuses it.

OpenAI: on by default, with breakpoints on newer models

OpenAI’s guide says prompt caching "is enabled by default for supported OpenAI models." No code change is needed.

GPT-5.6 changed the details. On GPT-5.6 and later, including the GPT-6 family, the minimum is 1,024 visible input tokens. OpenAI places a breakpoint at the end of the latest eligible message. An explicit mode lets you mark breakpoints yourself, up to four cache writes per request. The default cache lasts 30 minutes, and each read extends it.

These models also bill writes. GPT-6.1 Sol lists normal input at $2 per million tokens, a 30-minute write at $2.50 and a read at $0.10. The guide does not say outright whether default-mode writes are billed, but its default-mode example reports a cache_write_tokens count. Check your usage for it.

Earlier models charge nothing extra to write. A stable prompt_cache_key helps route related requests to the same cache, though OpenAI says it does not guarantee a hit. Cached tokens show in usage.input_tokens_details.cached_tokens.

Anthropic: cache_control breakpoints and two lifetimes

Anthropic caches only when you ask, in one of two ways.

Automatic mode. Add one cache_control field at the top level. The breakpoint goes on the last cacheable block and moves forward as a conversation grows.

Explicit breakpoints. Put cache_control on up to four content blocks. You can cache a long system prompt and a document set as separate layers, so changing one keeps the other.

The cache covers tools, then system, then messages. A change to tool definitions invalidates everything after them. Changing thinking settings or adding an image breaks the message cache too.

Writes cost 1.25 times input for the 5-minute cache and 2 times for the 1-hour cache. Reads cost a tenth of input on most models, less on the newest. Anthropic’s pricing page says caching "pays off after one cache read for the 5-minute duration," or two reads for the hour. Each hit resets the 5-minute clock free.

The minimum is 512 tokens on current models. Caches are separate per workspace on the Claude API. Responses report cache_creation_input_tokens and cache_read_input_tokens.

Google Gemini: implicit caching, plus paid storage on older models

Google runs two kinds of caching.

Implicit caching is on by default for Gemini 2.5 and newer. Google passes on the saving when a request hits, with no guarantee that it will. Its tips match the others: put large, common content first and send similar requests close together. The minimum is 4,096 tokens on Gemini 4 Argon and 3.8 Flash.

Explicit caching, also called context caching, is the manual kind. You upload content once as a cache object, set a lifetime and point later requests at it. Google pitches it for when you want guaranteed savings. It is in beta for Gemini 3 and 2.5 models and is not supported on Gemini 4 Argon or later.

Explicit caching adds storage, billed for every hour the cache exists, used or not. On Gemini 3.8 Flash that is $0.50 per million tokens per hour through December 31, 2026.

Teams on Gemini 4 Argon get implicit caching only: no storage bill, and less control over hits.

How to structure prompts so prompt caching hits

Stable content first, variable content last. Order each request like this.

1. Tool definitions, identical and in the same order every time.
2. System instructions. No date, user name or request ID here.
3. Shared reference material, such as a policy handbook or style guide.
4. Conversation history. Append new turns. Do not rewrite old ones.
5. Per-request content: retrieved passages, the question, the timestamp.

The usual cache breaker is a small dynamic value near the top. "Today is November 6" at the start of a system prompt breaks the cache daily. A request ID breaks it on every call. Reordering tools, switching models and loading documents in a different order do the same.

Agents gain the most, since they resend the whole conversation on every step. We covered that pattern in Claude Managed Agents dynamic workflows.

To confirm prompt caching is working, log the cached token counts for a day of real traffic. Zeros mean the prompt is too short, the prefix is changing or traffic is slower than the cache lifetime.

A worked example: one chatbot, three hit rates

The numbers here are our arithmetic, using rates checked October 21, 2026.

A member-services chatbot sends 8,000 stable tokens and 1,000 variable tokens per request and gets 400 tokens back. It handles 3,000 requests a month. Rates: $2 input, $10 output, $0.10 cached read and $2.50 cache write per million tokens. Claude Sonnet 5.5 and GPT-6.1 Sol both list exactly these. Gemini 4 Argon lists the same input, output and read rates through November 30, with no write charge.

Without caching, input is 27 million tokens ($54) and output 1.2 million ($12). The bill is $66. With caching, the 8,000 stable tokens are a read or a write on each request.

At 90%, the bill falls 60%. At 10%, prompt caching with write charges costs $6 more than no caching. Break-even is about one request in five hitting the cache.

Timing drives the hit rate. 3,000 requests a month is roughly one every five minutes in business hours, right at the edge of Anthropic’s 5-minute cache. OpenAI’s 30-minute default or Anthropic’s 1-hour option would catch far more. Our Claude Fable 5.1 cost analysis showed the same effect on a pricier model.

When prompt caching doesn’t help

Short prompts. Below 512 tokens on current Claude models, 1,024 on current OpenAI models and 4,096 on current Gemini models, nothing is cached.

Unique requests. Summarizing 500 different documents once each shares only the instructions.

Slow traffic. A form submitted once an hour outlives every default cache. With write charges, each request pays the premium and never collects.

Output-heavy jobs. A job that writes long drafts from a short prompt spends most of its money on output.

Idle explicit caches. In our arithmetic, a 500,000-token document set kept a full month on Gemini 3.8 Flash costs $180 in storage. Queried 200 times, caching saves $67.50 on input. Storage wins. Set short lifetimes.

Prompts that drift. A rotating date or reordered tools at the top turn every read into a write.

Cached tokens also still count toward the model’s context window.

Frequently Asked Questions