Async tool calling went generally available in OpenAI’s Responses API on September 3, 2026, alongside mid-turn steering and an opt-in replacement for context compaction in Codex. It received almost none of the coverage that the benchmark table on the same announcement page did.
That is the wrong way round. The benchmark numbers describe a model that most organizations cannot access yet. These three changes describe how agents behave, they are documented, and two of the three work today.
This piece covers what async tool calling actually does, the specific failure mode it removes, the constraints you need to know before designing around it, what mid-turn steering adds, how cross-context notes differ from compaction, the fact that none of it has been independently tested, and how to evaluate all three in an afternoon.
The short version: an agent that stalls while a tool runs is spending wall-clock time doing nothing, and for most agency workloads that stall is the dominant cost in how the product feels. Removing it changes user-perceived quality more than a few points of benchmark score ever will.
What async tool calling is
In the conventional loop, a model emits a tool call, the runtime executes it, and the model waits. Everything the model might have reasoned about in the meantime happens after the tool returns, in sequence.
Async tool calling breaks that dependency. OpenAI’s description: "Async tool calling lets the model continue working after it calls a tool, without waiting for that tool’s result." The result is matched back into the conversation later by its call_id.
The model can therefore issue a call to something slow, keep reasoning about the parts of the task that do not depend on it, and incorporate the result when it arrives.
The stall is the problem, and it is bigger than it looks
Consider a plausible agency workload: an agent that updates content across a headless CMS, triggers a rebuild, checks the resulting pages, and reports back.
That is an AI agent in the ordinary sense: a model with tools, running a multi-step task to completion. The tools in that loop have wildly different latencies. A content API write returns in a few hundred milliseconds. A static site rebuild takes forty seconds to four minutes. An image pipeline takes ten to sixty seconds. A deploy hook can take minutes. A third-party integration might time out entirely.
In a synchronous loop, the model does nothing during every one of those waits. If the run involves three rebuilds and a handful of image operations, the majority of wall-clock time is a model sitting idle while your user watches a spinner. The token cost of that run is unchanged, but the perceived quality of the product is set almost entirely by it.
Async tool calling lets the model plan the verification step while the build runs, draft the summary while images process, and queue the next content operation while the deploy completes. Nothing about the model got better. The loop stopped wasting time.
This is the same lesson visible from a different angle in the ARC-AGI-3 harness result, where a scaffolding change moved a score by 37 points on an unchanged model. Agent performance is substantially a scaffolding problem, and the scaffolding is the part you control.
The constraints, in full
These matter before you design around it, and they are easy to miss in the documentation.
- Responses API only. Tool calling for this model family requires it. If you are on an older endpoint, this is a migration, not a feature flag.
- GPT-6 Astra and later. Not backported to earlier models.
- Function and custom tools only. It does not work with hosted built-in tools.
- Incompatible with programmatic tool calling.
- Cannot be combined with parallel tool calls in multi-agent mode.
That last pair is the one that will bite an existing implementation. If your agent already fans out parallel calls across sub-agents, you cannot simply add async on top; you have to decide which concurrency model you are using.
There is also a design consequence that the documentation does not spell out. Once results arrive out of order, your prompt construction has to be robust to a tool result landing after the model has already reasoned past the point where it was requested. That is a correctness question, not a performance one, and it is worth writing tests for before it reaches a client.
Mid-turn steering
The companion feature lets you inject instructions over a WebSocket connection while a response is still being generated.
The obvious use is correction: a user notices the agent heading the wrong way and redirects it without waiting for the turn to finish and starting a new one. The less obvious and probably more valuable use is supervision. A monitoring layer can watch what an agent is doing and intervene on a policy violation mid-turn rather than after the fact, which is a meaningfully different safety posture from reviewing completed output.
For anyone building an approval workflow into an agent product, that is the piece to look at. It turns "review the result" into "intervene during the work," which is closer to how a human supervisor actually operates.
Cross-context notes, and why they are not compaction
The third change is in Codex and it is the most substantively new engineering idea in the release.
The problem it addresses is familiar to anyone who has run a long agent session. When the context window fills, the conventional fix is compaction: summarize everything so far into a compressed block and continue. Every compaction is lossy, and compactions stack. Twenty turns later the agent has a summary of a summary of a summary, and the specific detail it needs, why the previous fix failed, which file the error actually came from, is gone.
OpenAI’s alternative: the model "can keep notes across context windows, preserving accumulated details without repeatedly compressing them." Earlier context windows stay searchable rather than being replaced by their own summary.
The status is specific and worth quoting because it is unusually clear about what "shipped" means: "You can enable this experimental feature in your Codex config.toml, and it will become the default for Astra in the coming weeks." So it works today, behind a flag, with a promised default and no date.
There is one piece of outside corroboration for the mechanism. ARC Prize, evaluating the model on a completely different workload, singled out its note-keeping behavior and described the notes as remarkable for their "precision and information density." Games are not code migrations, and that is not a controlled comparison, but it is at least an observation from someone with no stake in the launch.
For agency work the target case is obvious: long refactors, framework migrations, and dependency upgrades across a large codebase, where the agent’s memory of what it already tried is the thing that decides whether the session succeeds.
None of this has been independently tested
Worth stating plainly, because the rest of this article is enthusiastic.
There is no third-party evaluation of async tool calling, mid-turn steering, or cross-context notes as of this writing. There are no published latency comparisons, no measured throughput improvements, no reproduction of any claim about them. What exists is OpenAI’s documentation, which describes capabilities rather than results.
Treat all three as documented capabilities to evaluate, not as reviewed features. Anyone telling a client that these deliver a specific speedup is inventing a number.
That is a separate matter from the model itself. Access to GPT-6 Astra is still restricted to a limited set of organizations, and enterprise administrators must enable it per workspace with it off by default. We went through the shipped-versus-announced split in what actually shipped with GPT-6 Astra.
How to evaluate this in an afternoon
If you have access, this is a bounded experiment rather than a project.
Instrument the stall first. Before changing anything, log the wall-clock time your existing agent spends between issuing a tool call and receiving the result, summed across a representative run. If that number is small, none of this matters to you and you can stop here. If it is more than a third of total run time, continue.
Pick your slowest tool. Not the most important one, the slowest one. Build the smallest possible harness that issues that one call asynchronously and gives the model something useful to do meanwhile.
Measure wall clock and tokens separately. Async should improve the first and leave the second roughly unchanged. If token usage rises significantly, the model is doing speculative work that gets discarded, and that is a real cost you need to see.
Write the out-of-order test. Force a tool result to arrive late and confirm your agent handles it correctly rather than plausibly.
For cross-context notes, run one long session both ways. Take a real migration task that currently overruns the context window, run it with compaction and with notes enabled, and compare not the output quality but how often the agent repeats work it already did. Repeated work is the symptom compaction produces and the one notes are meant to remove.
The pattern underneath
Three separate features in one release all point at the same idea: stop throwing away state.
Async tool calling stops discarding the model’s attention while it waits. Cross-context notes stop discarding accumulated detail when the window fills. Mid-turn steering stops discarding a turn in order to correct it.
That is a useful lens to carry into your own builds regardless of which model you use, because most agent implementations discard state by default. Each model call is treated as independent because that is the simplest thing to write, and the loop compensates by stuffing an ever-growing transcript into the prompt, which costs input tokens and degrades as it grows.
The cheapest performance gain available in most agent builds is not a model upgrade. It is noticing what your loop is throwing away between turns.
One caution to attach to that enthusiasm. State the model holds and you cannot inspect is convenient and unauditable. If your agent’s behavior depends on internal state you have no view into, you cannot reproduce a failure, you cannot debug it, and you cannot explain to a client why it did what it did. That trade is acceptable for a benchmark run and much less acceptable for a workflow touching a customer’s data. Keep the state you can read.
Frequently Asked Questions
What is async tool calling? A change that lets a model continue reasoning after issuing a tool call rather than waiting for the result, with the result matched back into the conversation later by its call_id.
When did it become available? September 3, 2026, generally available in OpenAI’s Responses API alongside the GPT-6 Astra launch.
Which models support it? GPT-6 Astra and later. It has not been backported.
Does it work with the Chat Completions API? No. Tool calling for this model family requires the Responses API.
Can I use it with built-in tools? No. It supports function and custom tools only, not hosted built-in tools.
Can I combine it with parallel tool calls? Not in multi-agent mode, and it is also incompatible with programmatic tool calling. If your agent already fans out parallel calls, you need to choose one concurrency model rather than layering them.
Does async tool calling reduce my token bill? Not directly. It reduces wall-clock time. If anything, watch for token usage rising if the model does speculative work that gets discarded once the tool result arrives.
What is mid-turn steering? The ability to inject instructions over a WebSocket connection while a response is still being generated, so a user or a monitoring layer can redirect an agent during a turn rather than after it.
What are cross-context notes? A replacement for compaction in Codex. Instead of summarizing a full context window into one compressed block, the model keeps running notes across windows and earlier windows remain searchable.
Are cross-context notes on by default? No. They are enabled through Codex’s config.toml today. OpenAI says the feature will become the default for Astra in the coming weeks, without giving a date.
Has any of this been independently benchmarked? No, as of this writing. There are no third-party latency or throughput results for any of the three features. Evaluate them yourself rather than relying on a published figure.
Do I need GPT-6 Astra access to try it? Yes for async tool calling and mid-turn steering, and Astra access is currently limited to a set of organizations with enterprise administrators enabling it per workspace, off by default.
Is the underlying idea portable to other models? The specific APIs are not. The pattern is: identify what your agent loop discards between turns, whether that is idle time, accumulated detail, or a whole turn thrown away to correct it, and stop discarding it. That applies whatever model sits underneath.
