What Is a Context Window? How Much an AI Model Can Read at Once, and What It Costs

A context window is the total amount of text an AI model can work with in a single request, counted in tokens. It holds your instructions, any files you attach, the conversation so far and the model’s reply. Anything outside the window does not exist for the model on that request.

Our guide to what a large language model is gave the context window two paragraphs. This post goes further. It covers what counts toward the window and how big windows are on current models. It explains why filling one costs more and can lower quality. It then covers how context differs from "memory" and how agencies can keep inputs small with long PDFs, site crawls and chat history.

The short version: the main models from Anthropic, OpenAI and Google each take about one million tokens. You rarely want to use all of it. Every token in the window is billed on every call. Some vendors charge more past a threshold. And models get worse at finding the right detail as the input grows. Send the model the part of the material it needs, put the question at the end and start fresh often.

What counts toward the context window

On most current models the window is one budget shared by input and output. Anthropic’s context windows guide defines it as "all the text a language model can reference when generating a response, including the response itself."

It holds:

• The system prompt and any standing instructions.
• Every message so far, from you and from the model.
• Attachments: documents, PDFs, images and pasted text.
• Tool definitions and tool results, when the model can search, read files or call an API.
• The reply, including any hidden reasoning or "thinking" tokens.

OpenAI’s conversation state guide says the same: the limit covers input, output and reasoning tokens. Reasoning tokens never show in the answer, yet they take space and are billed as output.

For the unit itself, see What Are Tokens? Counts differ by vendor and even by model. Anthropic’s glossary says Claude 4.7 and later produce about 30 percent more tokens for the same text than earlier Claude models.

Tokens in vs tokens out

Input and output usually share the context window, but they have separate limits and separate prices.

Input is everything you send. On current flagship models it can run close to one million tokens.

Output has its own, much smaller cap. It is 128,000 tokens on current Claude and OpenAI flagships. The Gemini models in our table allow 65,536 by default.

Google draws the line in two ways. Gemini 4 Argon has one unified window of 1,048,576 tokens for input and output combined. Gemini 3.8 Flash lists 1,048,576 input tokens and a separate 65,536 output limit. On a unified window, a long reply eats into the room left for input.

Output usually costs several times more per token. Our post on input vs output tokens covers that gap.

Context window sizes as of November 2026

This is the one section of this post that will date. Figures come from each vendor’s model documentation, checked on October 21, 2026.

Sources: Anthropic’s models overview and pricing page, OpenAI’s model pages, and Google’s Gemini 4 Argon page and pricing page. Our Gemini 4 Argon launch post and Claude Haiku 5.5 post cover those models in more detail.

Chat apps are separate products, and their limits can differ from the API by plan. Anthropic notes that chat interfaces can drop the oldest turns on a rolling "first in, first out" basis.

Why a full context window costs more

Two cost effects stack.

1. You pay for the whole window on every call. Each new turn resends the full history. OpenAI’s guide says that when you chain responses, "all previous input tokens for responses in the chain are billed as input tokens."

A hypothetical example: if each turn of a chat adds 2,000 tokens, turn 30 sends about 60,000 tokens. Across all 30 turns, you pay for roughly 930,000 input tokens.

2. Some vendors charge a higher rate past a threshold. OpenAI’s model pages say prompts over 272K input tokens are priced at 2x input and 1.5x output "for the full request." So a 300,000-token prompt to GPT-6.1 Sol costs about $1.20 in input instead of $0.60. Claude Haiku 5.5 switches from $0.10 to $0.50 per million input tokens past 100,000. Anthropic’s other current models have no surcharge. Its pricing page puts it this way: "A 900k-token request is billed at the same per-token rate as a 9k-token request." That holds across the full window.

Prompt caching softens the first effect when the start of your prompt repeats. Anthropic bills cache hits at a fraction of the base input price. Our post on Claude Fable 5.1 and cache hit rates shows how much that can change a bill. Cached tokens still count against the context window, though. Only the price changes.

How a long context window can lower quality

A widely cited study is Lost in the Middle: How Language Models Use Long Contexts by Nelson F. Liu and colleagues, first posted in July 2023 and accepted at TACL. The authors gave models 10, 20 or 30 documents, with the answer in one, and moved that document around. In their words, "performance is often highest when relevant information occurs at the beginning or end of the input context." Accuracy dropped when the answer sat in the middle.

Two details stand out. Take GPT-3.5-Turbo with 20 or 30 documents. With the key document in the middle, it scored lower than with no documents at all. And models sold as "extended context" were no better at using their input than their standard versions.

The paper tested 2023 models, including GPT-3.5-Turbo and Claude 1.3. Today’s vendors still say quality drops as input grows:

• Anthropic’s guide calls it context rot: "As token count grows, accuracy and recall degrade."
• Google’s long context guide says Gemini finds a single fact well. With several facts to find, the model "does not perform with the same accuracy."

We looked at the wider question of when more tokens help in When More Tokens Make AI Output Worse.

Context window vs memory

The context window is short-term. It exists only for one request. Anthropic’s glossary calls it the model’s "working memory." When the request ends, nothing in it carries over unless you send it again.

Memory is anything stored outside the model and loaded back in later. That can be a saved note, a file or a database row. Anthropic’s memory tool docs describe agents that record what they learn in files and read them "back on demand," which "keeps the active context focused on the current task."

Memory still has to enter the context window to be used. A memory feature saves you retyping, and the window stays the same size. Workspace features work the same way. Our post on Claude Projects and our explainer on Gemini Notebook cover two workspace products that keep files on hand across chats.

How RAG and summarization keep the context window small

Retrieval (RAG). Store the material in a search index. For each question, pull only the most relevant passages into the prompt. Our guide to retrieval-augmented generation explains the design, and What Are Vector Databases? covers the search side. In Liu and colleagues’ tests, going past 20 retrieved documents improved answers by only about 1 to 1.5 percent.

Summarization and compaction. Replace old material with a shorter version. Anthropic and OpenAI both offer this server side:

• Anthropic offers server-side compaction in beta for Claude 4.6 and later. It summarizes earlier turns so a conversation can continue past the limit. Its context editing feature can also clear old tool results.
• OpenAI’s compaction guide describes a threshold setting. When the token count crosses it, the server compacts the context automatically.

Summaries lose detail, so re-send exact text when wording matters.

Practical context window tips for agency work

Long PDFs. To ask about one table in a 300-page annual report, extract those pages first. Sometimes you must send the whole file. Anthropic’s prompting guide says to put long documents at the top and the question at the end. It reports that queries at the end can improve quality "by up to 30 percent in tests." Google gives the same advice. Ask the model to quote the relevant passages before it answers. Claude also caps PDF and image pages at 600 per request on 1M-token models.

Site crawls. A raw crawl is mostly navigation, footers and scripts. Strip those and convert pages to plain text or Markdown before they reach the model. Send one page per request for jobs like meta descriptions. For questions across the site, index the crawl and use retrieval.

Chat history. Start a new chat for each task. When a thread runs long, ask for a summary of decisions. Paste it into a fresh chat. Keep a standing project brief for each client.

Agents. Tool results fill windows fast. Use compaction or context editing, and log state to files.

Measure first. Count tokens on a few real inputs, multiply by volume and watch the price thresholds in the table above.

Frequently Asked Questions