Drew Breunig's AGNTCon talk maps four context failures and six fixes

Drew Breunig gave a short talk at AGNTCon this week titled "Context Fails & Fixes" and has posted the slides with a lightly edited transcript under the headline "The State of Context Engineering". The talk asks how context engineering emerged, where it has gone and where it might go, and argues that as model makers chase new capabilities they inadvertently create challenges that users and harness builders must overcome.

Breunig opens with his forthcoming book, The Context Engineering Handbook. When he first spoke to the O'Reilly team in June 2025, they asked how the book would avoid going obsolete in a few weeks. His answer was optimism: giving clear, consistent, non-noisy instructions about what to do and what not to do will forever be a challenge with AI, no matter how good it gets. People who are sentient and pass the Turing test still struggle to convey instructions to each other. Context engineering will stay hard, he says, but in ways specific to models, because models are not people.

The history starts at the end of 2024, the height of the "RAG is dead" moment. The hope was that a big enough context window would let you put everything in it, skip selection and skip agentic systems. Model makers aimed at long context and the window leapt from about 128,000 tokens to a million, starting with Gemini 1.5. Google teased a 10-million-token model, but Llama 4 Scout shipped one first. Inference was a challenge, Scout never really caught on, and in Breunig's words there was never another Llama. A million tokens stayed the norm. But more context meant more room for problems.

Breunig names four failure modes. Context poisoning is when something that should not be there, such as a hallucination or a bad fact, gets into the context and the model cannot get past it. Context distraction is when the context grows so long that the model focuses on it over its base knowledge. Context confusion is when extraneous facts unrelated to the problem influence the model in ways not fully understood; he gives the example of a model mixing work conversations with a project involving his son when memory is on, and says he turns memory off. Context clash is when tool calls, skills and other harness information conflict and work against the user.

His favorite poisoning example comes from Pokémon. In the original 1996 Game Boy game and in the 2004 Game Boy Advance remake FireRed, you must give a thirsty security guard a drink to get past. In the original it is lemonade; in FireRed it is iced tea. In the Gemini 2.5 technical report, researchers had the model play Pokémon, and several times it became convinced it needed tea even though it was playing the older game. The tea task was in its weights, so it poisoned its context; it spent hours trying to find or give the guard tea, and once tea was on its to-do list it could not get past it.

For distraction, Breunig polled the room: about half the audience said they let coding context fill to a million tokens. He starts looking at a reset around 200,000 or 300,000 tokens. He then describes the paper "Do as I Say, Not as I Do" by Carolina Camassa and Derek Shiller. A model is told to translate every question into Russian as its answer, then shown many examples in the chat history that contradict this, with answers in French. The matrix in the paper plots the number of bad examples against the model tested and shows how often the model sticks to the instruction. Breunig reads it as showing that every model is affected: Gemma 3 12B follows the instruction less than half the time after just a few contradictory examples, and Opus 4.6 takes many more examples but gives in too. Gemini 2.5 hit the same problem in Pokémon: an appendix to its technical report notes that once context grew significantly beyond 100,000 tokens, the agent began repeating actions from its history instead of synthesizing new plans, even though Google had promoted its million-token window and suggested two million were coming. Long contexts let you fit in more information, and more errors.

The remedy is a toolbox of six techniques that, he says, has stayed the same even as new challenges emerged. Context augmentation brings in relevant information through RAG, skills and similar means. Tool loadout selects which tools and skills to expose; he recalls an enterprise team whose agents suddenly performed terribly because their software loaded 600 skills for every engineer at startup. Context quarantine uses subagents or peer agents that start with a blank history in the same environment. Context pruning removes what is no longer needed; Claude Code and Codex do this, sometimes unnoticed, as early stages of compaction strip out unneeded tool results. Context offloading stores information in the environment, such as Markdown files or a database. Context summarization is what everyone calls compaction. These tools, he says, have increasingly moved into the harness.

To show how simple the patterns are, he builds a demo in DSPy. An agent is a while loop: a ReAct module takes a request, returns a response and is given tools that are plain Python functions. The demo uses Claude Sonnet 4.5 as the big model and a tiny, fast decision model called Jev as the small one, though a small LLM would do. The basic agent is about a dozen lines. Loadout is a function that asks the small model, tool by tool, whether each is needed for the request and returns the top-ranking subset. Quarantine exposes a subagent as a tool, starting from a blank history. Pruning steps through the history asking the small model whether each turn is still needed, and cuts it if not; he warns to mind prompt caching. Compaction rewrites the history by summarizing old turns and starting fresh from the summary. A variation he likes lately is to check what matters before compacting: a decision model makes a first relevance pass, and only the smaller subset is summarized. Together, the four fixes add about 16 lines to one loop. The visible text ends partway through the demo, so the closing sections of the talk are not covered here.

Key facts

Drew Breunig's AGNTCon talk "Context Fails & Fixes" names four long-context failure modes: poisoning, distraction, confusion and clash.
Gemini 2.5's Pokémon run illustrates both: it repeatedly chased a tea item from the FireRed remake while playing the older game, and past roughly 100,000 tokens of context it began repeating actions from its history.
In the Camassa and Shiller paper, Gemma 3 12B follows a translate-into-Russian instruction less than half the time after just a few contradictory examples; Opus 4.6 needs many more but is affected too.
The toolbox has six techniques: augmentation, loadout, quarantine, pruning, offloading and summarization (compaction), increasingly built into the harness.
A DSPy demo of about a dozen lines of agent loop gets four of the fixes (loadout, quarantine, pruning, compaction) with about 16 lines of changes.

Why it matters

The talk is a compact account of how the field's thinking shifted. At the end of 2024 the pitch was to stuff everything into a million-token window and let the model sort it out. Breunig's argument is that long context brings its own failures, and that the fixes are ordinary harness engineering rather than waiting for bigger windows. His case that clear, consistent instructions will stay hard however capable models get is the thread that ties the talk together.

Who it affects

Anyone building or running agents and coding assistants. The poll at the talk, in which about half the room let coding context fill to a million tokens, suggests many practitioners do not reset sessions early. Harness builders get a six-item checklist; ordinary users of coding tools get a rule of thumb from Breunig's own habit of starting to consider a reset around 200,000 to 300,000 tokens.

How to use it

Treat the six techniques as a menu: augment with relevant material, filter tools and skills through a loadout step, hand side tasks to subagents with a blank history, prune stale tool results, offload information to files or a database, and compact when needed. In the demo, a small fast decision model does the filtering, and tools are plain Python functions in a DSPy ReAct loop. Breunig's favored variation is a relevance pass before compaction so the summary covers only what matters. He also advises minding prompt caching when pruning. His 200,000 to 300,000 token reset point is a personal habit, not a rule.

How solid is it

This is a talk transcript and opinion piece, not a study. The measured results it cites come from other sources as Breunig describes them: the Gemini 2.5 technical report and the Camassa and Shiller paper. The 600-skills story is an anecdote about one enterprise team. The visible text is truncated partway through the demo, so the closing arguments are not covered here.

Risks and caveats

The Pokémon and Camassa and Shiller examples are described secondhand, and Breunig himself calls the paper's experiment a little contrived. No exact Opus 4.6 figures are given. The demo relies on a small decision model for pruning and loadout; Breunig notes you can be as rough or as elegant as you like, but pruning can interact with prompt caching. The author also has a book coming, The Context Engineering Handbook, so he has a stake in the topic.

“Long contexts let you fit in more information, but they also let you fit in more errors.”

— Drew Breunig, "Context Fails & Fixes" talk at AGNTCon