A batch API lets you send thousands of AI requests in one go and collect the answers later, usually within 24 hours, at half the normal price. OpenAI, Anthropic and Google all offer one, and all three charge 50% of their standard rates for it.
Our guide to what AI actually costs mentions batch processing in one line. This post covers how the three batch services differ, the newer Flex tiers, which jobs fit, how to stack batch with caching, and a worked cost example.
The short version: if nobody is waiting on the answer, use the batch API. Alt text for an image library, tags for an archive, survey summaries and page translations all fit. Chat, search and anything a person watches load do not. Add prompt caching on top and the bill for a large job can drop by two thirds or more.
How a batch API works
With a normal API call, you send a request and get an answer in seconds. With a batch API, you bundle many requests into one job, submit it and come back later.
The steps are the same at all three vendors:
- Build the job. Each request gets your own ID, the model name and the usual prompt. At OpenAI and Google this is a JSONL file, one request per line. Anthropic takes a list of requests in one API call.
- Submit it. The vendor queues the job and runs it when it has spare capacity.
- Collect results. Download the output and match each answer to its request by your ID.
That last step matters. OpenAI’s batch guide says the output order "may not match" the input order. Always match by ID, never by line number.
The discount is the price of letting the vendor decide when your work runs. At OpenAI, batch work also has its own rate limits, so a big batch does not use up the limits your live chatbot needs.
Batch API options compared, as of October 2026
All figures below come from each vendor’s documentation, checked on October 21, 2026. Limits change, so check before you build.
| Detail | OpenAI Batch API | Anthropic Message Batches | Google Gemini Batch API |
|---|---|---|---|
| Price | 50% off | 50% off | 50% off |
| Turnaround | Within 24 hours; 24h is the only window | Most within 1 hour; expires at 24 hours | Target 24 hours; expires at 48 hours |
| Size per job | 50,000 requests or 200 MB | 100,000 requests or 256 MB | Inline under 20 MB, or a file up to 2 GB |
| Results kept | 30 days | 29 days | 6 weeks |
| Models | “Most models,” one model per file | All active models | A range of Gemini models |
| Caching | Cached input rate listed for batch | Supported; stacks with batch discount | Supported |
Sources: OpenAI’s batch guide, Anthropic’s batch processing docs and Google’s Gemini batch docs.
At list price, the mid-tier models line up. Claude Sonnet 5.5 and GPT-6.1 Sol cost $2 per million input tokens and $10 output, or $1 and $5 in batch. Gemini 4 Argon matches that through November 30, then doubles to $2 and $10 in batch from December 1, per Google’s pricing page.
What each vendor’s batch API does differently
OpenAI. The batch API covers more than text. It accepts requests to the Responses, Chat Completions, Embeddings, Moderations and image generation endpoints. Each file can hold requests for one model only. Each model also has a cap on how many tokens you can have queued at once, shown in your account’s limits page. If a batch runs out of time, finished requests are kept and billed. The rest are canceled and listed in an error file.
Anthropic. Anthropic’s Message Batches API takes the most requests per job. Its docs say most batches finish within an hour. Almost any Messages request works in a batch, including images, tool use and extended thinking. Streaming and Fast mode do not. You are not billed for requests that error, are canceled or expire. Batch is not available for Managed Agents sessions, which our post on Claude Managed Agents workflows covers.
Google. The Gemini batch API takes the largest files, up to 2 GB, and keeps results longest. Jobs can wait up to 48 hours before they expire.
Flex tiers: half price without the queue
OpenAI and Google now sell a second discounted option called Flex. You send a normal request and get a normal answer, just slower, at the batch price.
OpenAI Flex is in beta. You set service_tier to flex on a request. The Flex guide says tokens are "priced at Batch API rates," and model availability is limited. Requests can time out or return a 429 error when capacity is short. OpenAI says you are not charged when that happens.
Google Flex is in Preview, also at 50% of the standard rate. Google’s Flex docs list a 1 to 15 minute target. Flex requests can be pushed aside for higher priority traffic, and Google does not fall back to the standard tier for you. You need your own retries. Flex also counts against your normal rate limits.
Anthropic lists no Flex tier on its pricing page as of October 2026. Its discount options are batch and caching.
Flex suits jobs that run in steps, where each step needs the last answer. In a batch, each step would wait in the queue.
Good fits: work nobody is waiting on
If the result can arrive tomorrow morning, the job fits. Common ones for agencies, nonprofits and associations:
Alt text for an image library. A media library with 15,000 images and no descriptions is a classic batch job. Send each image with your alt text rules, collect drafts the next day, then have staff review them. Our web accessibility guide covers why that review step still matters.
Tagging an archive. Years of articles, resources or journal abstracts can be sorted into your current taxonomy overnight. Send the tag list and rules with each item, and ask for tags plus a short summary.
Summarizing survey responses. Member surveys with thousands of open-text answers fit well. Ask for a theme label and a one-line summary per response, then count the themes yourself.
Translating pages. Translating a resource library into Spanish is a large, one-time job. Batch it, then have a fluent reviewer check the pages that matter most.
Bad fits: anything a person is waiting for
Some jobs should never go through a batch API.
- Chat and member help. A visitor will not wait an hour for an answer.
- Search and form replies. Anything that loads while someone watches the page.
- Agent loops. Each step depends on the last. Use Flex or the standard tier.
- Hard deadlines. Batches can expire, and Anthropic warns that heavy demand can mean more requests expire.
How to stack batch with prompt caching
Most batch jobs repeat the same opening text in every request: your instructions, tag list and style rules. That is what prompt caching is for.
Anthropic’s pricing page says its cache multipliers "stack with other pricing modifiers, including the Batch API discount." Its batch docs add that users "typically experience cache hit rates ranging from 30% to 98%." Because batches can run longer than five minutes, Anthropic suggests the 1-hour cache for batch work. A 1-hour cache write costs twice the base input price.
OpenAI’s pricing page lists a cached input rate in its Flex/Batch table: $0.05 per million tokens for GPT-6.1 Sol. Google says context caching is supported for batch requests, and its pricing page lists a batch cache rate at half the standard one.
Two habits raise the hit rate. Put the fixed text first and the item-specific text last. Keep the fixed text identical, byte for byte, across every request in the job.
A worked example: tagging a 12,000-article archive
Here is our arithmetic for one job on Claude Sonnet 5.5 at October 2026 list prices. GPT-6.1 Sol has the same standard and batch rates, so the first two rows hold there too.
Each request sends 2,000 tokens of fixed instructions and tags, plus a 1,200-token article. Each answer is about 200 tokens. Across 12,000 articles, that is 24 million fixed tokens, 14.4 million article tokens and 2.4 million output tokens.
| Setup | How it is priced | Cost |
|---|---|---|
| Standard API | 38.4M input at $2, 2.4M output at $10 | $100.80 |
| Batch API | 38.4M input at $1, 2.4M output at $5 | $50.40 |
| Batch plus caching | Fixed text: 90% cache hits at $0.05, 10% 1-hour writes at $2. Articles at $1. Output at $5. | About $32.30 |
The batch API alone halves the bill. Caching the fixed text takes it to about a third of the standard price. That depends on the hit rate. Each 1-hour cache write costs twice the batch input price, so at a 50% hit rate the job costs about $51, slightly more than batch alone. On this job, caching pays off only above roughly a 51% hit rate.
Model choice still beats both discounts. The same job on Claude Haiku 5.5 in batch, at $0.05 input and $0.25 output, costs about $2.50 before caching. Test the small model first; our Claude Haiku 5.5 post covers which jobs it handles. If outputs are long, remember that output tokens cost five times input, as our input vs output tokens guide explains.
Before you send your first batch
- Run 50 requests the normal way first. Fix the prompt, and log the tokens to estimate the full job.
- Give every request a clear ID. Use your CMS item ID so results map back cleanly.
- Check for missing results. Resubmit expired or failed requests.
- Plan the review. Alt text, translations and summaries still need a person to check them before they go live.
Frequently Asked Questions
What is a batch API?
It is a way to send many AI requests as one job and get the results later, usually within 24 hours. OpenAI, Anthropic and Google all charge half their normal rates for it.
How much does a batch API save?
As of October 2026, all three vendors charge 50% of standard prices on both input and output tokens. Adding prompt caching can cut a large job further, to about a third of the standard cost when most of each request repeats.
How long does a batch take?
OpenAI completes batches within 24 hours. Anthropic says most finish within an hour, with a 24-hour limit. Google targets 24 hours and expires jobs after 48 hours.
What happens if a batch does not finish in time?
The unfinished requests expire. Finished results stay available. OpenAI bills only the completed requests, and Anthropic does not bill expired ones. Check the error or results file and resubmit what is missing.
What is the difference between Batch and Flex?
Batch is a queued job you collect later. Flex is a normal request that may run slowly or fail when capacity is short. OpenAI and Google price Flex at the batch rate. Anthropic has no Flex tier listed as of October 2026.
Can I use prompt caching with a batch API?
Yes. Anthropic says the discounts stack, and it suggests the 1-hour cache for batch work. OpenAI lists cached input rates for batch, and Google supports context caching in batch jobs.
Can a batch API handle images?
Anthropic’s batch docs list vision among the supported features. OpenAI’s batch API takes the same Responses and Chat Completions requests used for image input. Test a few images through each vendor before a full alt text run.
Should I use a batch API for my website chatbot?
No. Visitors expect answers in seconds. Use the standard tier for chat, and save batch for back-office jobs such as tagging, summaries and translation.
