Pi details how it compacts long coding conversations

Large language models can only see a limited context window at a time, and a coding agent's conversation keeps growing with every turn: the system prompt, loaded files such as AGENTS.md, tool definitions, user messages, assistant replies, tool calls and their results all pile up. Anyone who has had a long session in a coding agent like Pi, Claude Code or Codex has hit this wall and triggered a compaction. Eventually the history exceeds the context window and the next request fails with an error such as "Request exceeds the maximum size." At that point there are two options: start a new, empty conversation, which discards the history, including prior decisions and unresolved work (though the post notes this might still be worth doing, since LLM output quality tends to decrease as context grows), or compact the conversation into a smaller representation that keeps the session going. A post explains exactly how Pi, a coding agent, implements its own version of compaction.

In theory, compaction could be done with a deterministic function that keeps part of a conversation and drops the rest. In practice, real implementations, Pi's included, use an LLM request to summarize the history instead: compaction replaces part of the conversation with a compressed representation, freeing up room for more messages and tool calls.

Pi triggers compaction automatically once the context nears the size of the context window, and it can also be forced with the /compact command. Pi checks whether auto-compaction is needed after each turn ends; until then, every request just extends the existing prompt and can reuse its cached prefix. If a request overflows the context window in the middle of a turn, Pi can also compact mid-turn to recover.

When Pi compacts, it keeps a number of the most recent messages completely unchanged, governed by a configurable token budget that currently defaults to 20,000 tokens, which the post says comes out to roughly 5 to 20 turns. Everything before that cut point gets extracted, serialized and handed off to be summarized.

That summarization step is a separate, standalone LLM request rather than a continuation of the ongoing conversation, designed to work like a handoff briefing from one shift to the next: keep only what's still important context for the next request, and drop the rest. Its system prompt differs from the one used for normal coding turns: instead of telling the model "you are an expert coding assistant," Pi tells it "you are a context summarization assistant." Its user message asks for "a structured summary of this conversation branch for context when returning later," with the prompt specifying sections for goal, progress and key decisions. Because this request does not carry the prior conversation history with it, Pi can route it to a different LLM model without adding to the cost of the main conversation.

The resulting summary is appended to the session as a compaction entry, sitting ahead of the retained recent turns and the next new user message. Pi stores it as plain text rather than a proprietary format, which keeps the compacted context readable and portable: because the summary is just text, a user can switch which LLM model Pi is using mid-session and keep working from the same summary.

Compaction has a cost, though. Prompt caching, which lets LLM providers charge less for the parts of a request that exactly match a previous one, requires an exact prefix match. Once compaction runs, the retained recent turns sit behind a new summary instead of the old history, so their previously cached prefix can no longer be reused, and the first request after compaction loses that discount. Caching benefits return on subsequent requests once the new, post-compaction prefix itself starts getting reused.

The post closes by noting that Pi's default compaction behavior isn't fixed: because Pi is described as extensible, a user who wants a different approach can ask Pi to build an extension with a custom compaction prompt.

Key facts

Pi retains recent messages unchanged using a configurable token budget that currently defaults to 20,000 tokens, which the post says comes out to roughly 5 to 20 turns; everything older than that cut point gets summarized.
Compaction runs as a separate, standalone LLM request whose system prompt swaps "you are an expert coding assistant" for "you are a context summarization assistant," and whose user prompt asks for a structured summary covering goal, progress and key decisions.
Because the compaction request drops the prior conversation history, Pi can send it to a different LLM model without adding to the cost of the ongoing session.
Pi checks for auto-compaction after each turn ends but can also compact mid-turn if a request overflows the context window; it can also be triggered manually with the /compact command.
Compaction breaks prompt caching because the retained turns now follow a new summary instead of the old history, so the first post-compaction request cannot reuse the previously cached prefix, though caching resumes on later requests.

Why it matters

Every coding agent built on an LLM runs into the same wall: conversations keep growing until they blow past the model's context window, and what happens next decides whether a long session degrades gracefully or just breaks. This post is useful because it doesn't wave at "we summarize the conversation" the way most agent documentation does; it gives the actual trigger conditions, the actual token budget, and the actual wording of the system and user prompts Pi sends to do the summarizing. That level of detail is rare for a mechanism most coding-agent users only ever see as a brief "compacting" message.

Who it affects

Anyone who has used a coding agent like Pi, Claude Code or Codex for a long session and watched it compact. It's also relevant to developers building their own agents on top of LLM APIs and running into the same context-window ceiling, since Pi's approach, a standalone summarization request with a distinct system prompt, a configurable retention budget, and plain-text storage of the result, is a concrete pattern they can compare their own agent against.

How to use it

Pi users can trigger compaction manually with the /compact command instead of waiting for it to fire on its own, and should expect the first request right after a compaction to lose the benefit of prompt caching, since the retained turns then sit behind a new summary. The post also notes that Pi's compaction isn't fixed: because Pi is extensible, a user who wants different behavior can ask Pi to create an extension with a custom compaction prompt rather than being stuck with the default. No pricing or licensing terms for Pi itself are given in the post.

How solid is it

The claims come from a single technical post written in first person plural about Pi, with implementation detail, exact system-prompt wording, the token-budget figure, that reads as though it comes from people with first-hand knowledge of Pi rather than an outside observer. That said, the piece carries no named author, though it is dated 13 Aug 2026 and its footer names the company as Earendil Inc.; specifics like the 20,000-token default are nonetheless explicitly described as configurable and could already have changed. The post also gives no real-world example of what an actual compaction summary looks like, so there is no way to check how much detail Pi's summarization step keeps versus loses in practice.

Risks and caveats

The post names Claude Code and Codex only as other coding agents that also trigger compaction; it does not describe or compare how their compaction mechanisms actually work, so none of Pi's specifics should be assumed to carry over to them. The post is dated 13 Aug 2026, and since compaction is explicitly configurable, a reader trying to match these numbers in their own Pi setup should verify them against the current product rather than treat them as fixed.

“Instead of telling the LLM "you are an expert coding assistant", we tell the LLM "you are a context summarization assistant."”

— the Pi compaction post