OpenAI ships better prompt caching for GPT-6
OpenAI announced, on September 22, 2026, an improved prompt caching system built into the GPT-6 family. GPT-6 is meant to run persistent agents that work for hours on tasks such as refactoring a codebase or producing a researched document, making a long chain of API requests that carry forward the same instructions, tool definitions and context. OpenAI caches that shared context so it does not have to be reprocessed on every call, and gives developers discounts of up to 90% on cached input tokens. With GPT-6, the company says it now delivers higher cache hit rates by default and gives cache discounts for eligible shared prefixes reused within a 30-minute window.
Alongside the higher default hit rates, OpenAI introduced a Prompt Caching Dashboard that shows how much of an application's input is served from cache and lets developers track hit rates over time and compare cached against uncached tokens. A companion prompt caching diagnostics tool explains an unexpected cache miss by comparing a request against a recent response, naming what changed (model, tools, settings or input) and estimating how many tokens were affected. Developers can also set explicit cache breakpoints to choose which prompt prefixes get reused, following a refreshed prompt caching guide that covers how long a cached prefix stays eligible.
A separate change lets developers on GPT-6 models raise or lower reasoning effort between requests, for a harder task or a routine follow-up, by appending a configuration_update, without breaking the cache the way changing request-level reasoning effort would. OpenAI also published guidance for keeping cache intact as tools and instructions change: keep tool definitions, schemas and ordering stable, use allowed_tools or tool_choice set to none instead of removing tool definitions outright, and append new instructions as new developer messages rather than editing earlier ones. A prewarming feature lets an application load shared instructions, tool definitions or reference material into the cache ahead of time, before a user's first question arrives, moving that processing out of the user's wait time.
The announcement carries customer figures. Mario Rodriguez, Chief Product Officer, said prompt caching plays a critical role in helping GitHub Copilot run at scale, and that over the past several months the share of prompt tokens requiring fresh processing across billions of requests to OpenAI models fell by more than 50% relative to their previous baseline. Arian Hanifi, Chief Technology Officer, said the new diagnostics and dashboard raised cache hit rates by a few percentage points and cut costs by 20%. Bin Fan, an agent team lead, said reliable caching is fundamental to the economics of long-running agents such as Manus, and that after working with OpenAI's engineering team on breakpoint placement, his cache hit rate went from roughly 85% to consistently above 90% in under a week. Eugene Mikhantyev, an AI engineer, said moving session agents to explicit cache breakpoints raised evaluation cache hit rates from 83% to 91% in under a week, cutting cache writes by roughly two-thirds and inference costs by 36%. The source does not state which companies employ Hanifi, Fan or Mikhantyev.
Key facts
- OpenAI's improved prompt caching for the GPT-6 family gives higher cache hit rates by default and applies discounts of up to 90% on cached input tokens for shared prefixes reused within a 30-minute window.
- New tools include a Prompt Caching Dashboard for tracking hit rates and a diagnostics tool that identifies the cause of a cache miss and estimates how many tokens it affected.
- Developers can now raise or lower reasoning effort between GPT-6 requests via a configuration_update without breaking the cache, and can set explicit cache breakpoints to control which prefixes get reused.
- GitHub Copilot's Chief Product Officer Mario Rodriguez said the caching cut the share of prompt tokens needing fresh processing by more than 50% across billions of requests over the past several months.
- AI engineer Eugene Mikhantyev reported that moving to explicit cache breakpoints raised evaluation cache hit rates from 83% to 91% in under a week, cutting cache writes by roughly two-thirds and inference costs by 36%.
Why it matters
Agentic applications on GPT-6 make long chains of API calls that repeat the same instructions, tool definitions and context. Reprocessing that context on every call costs time and money, so how well a provider caches it directly sets the latency and the bill for anyone running an agent for hours at a stretch. OpenAI's own figure, discounts of up to 90% on cached input tokens, shows how much is at stake in the hit rate alone.
Who it affects
Developers building persistent or agentic products on the GPT-6 API are the direct audience: the dashboard, the diagnostics tool and the explicit breakpoints are all developer-facing controls. The announcement leans on named individuals at customer organizations, GitHub Copilot's Chief Product Officer Mario Rodriguez, plus a Chief Technology Officer, an agent team lead who cites the agent platform Manus, and an AI engineer, though the source names only Rodriguez's product (GitHub Copilot) and leaves the other three speakers' employers unstated.
How to use it
The Prompt Caching Dashboard tracks hit rates and compares cached against uncached tokens; the diagnostics tool compares a request to a recent response to explain a cache miss and size its token impact. Explicit cache breakpoints let a developer choose which prefixes to keep reusable, per the refreshed prompt caching guide. On GPT-6, reasoning effort can be changed between requests with a configuration_update without invalidating the cache. OpenAI also recommends keeping tool definitions, schemas and ordering stable, using allowed_tools or tool_choice: none instead of removing tools, appending new instructions as later developer messages, and prewarming shared context before a user's first request.
How solid is it
This is OpenAI's own product announcement, published on its blog on September 22, 2026, alongside the GPT-6 family. The customer results are self-reported testimonials attributed to named individuals and job titles, not an independently verified benchmark, and the source gives no methodology for how each company measured its before-and-after hit rates.
Risks and caveats
Three of the four quoted customers, a Chief Technology Officer, an agent team lead and an AI engineer, are not tied to a named employer in the source, so their figures cannot be checked against a specific company's public claims. The article gives no separate version number or rollout date for the caching system beyond pairing it with the GPT-6 family's own launch, and it does not state what the previous default cache hit rate was, only the percentages customers report for their own workloads.
“Working with OpenAI, we moved our session agents to explicit cache breakpoints. In under a week, cache hit rates on our evaluations rose from 83% to 91%. This meant fewer cache writes and lower inference costs with the same workload; the cache writes fell by roughly two-thirds, and inference costs by 36%.”
— Eugene Mikhantyev, AI Engineer