Glonce

Coca-Cola compresses LLM agent tool outputs into latent tokens

researchThe Glonce editorsHugging Face🔥60Glonce rating60Impact45Novelty65Relevance100Surprise30Players10Credibility75
Coca-Cola compresses LLM agent tool outputs into latent tokens

LLM agents usually paste raw tool outputs straight into their context. Over many steps, the authors say, those long and noisy observations add substantial context overhead and dilute the agent's attention to the information that matters for the task. Existing token pruning and summarization methods, they argue, stay tied to discrete text, which limits how far they can compress.

The paper challenges the assumption that an agent has to read tool observations as explicit text tokens. Its answer is Coca-Cola, a continuous latent context compression model for LLM agents. Coca-Cola compresses a tool observation into latent tokens and embeds them in the input embedding space of the agent, so the agent consumes the compressed form directly.

To train the compressor, the authors introduce Action-Preserving Training. It aligns the agent's next-action distributions under full observations and under compressed ones. The aim is for the compression model to learn action-sufficient latent tokens: ones that keep the information the agent needs to decide what to do next.

The experiments use web-search agents on four QA benchmarks and two model families. The authors report that Coca-Cola reduces the agent's token consumption by 70.01%-89.24% and end-to-end runtime by 24.45%-55.97%, while consistently improving task performance. They conclude only that the results suggest Coca-Cola offers a promising approach for context compression in agents. Code is available on GitHub at https://github.com/THU-Agent/Coca-cola.

Key facts

  • Coca-Cola is a continuous latent context compression model: it turns tool observations into latent tokens placed in the agent's input embedding space, rather than keeping them as text.
  • Action-Preserving Training aligns the agent's next-action distributions under full and compressed observations, so the latent tokens keep decision-critical information.
  • On web-search agents across four QA benchmarks and two model families, the paper reports token consumption down by 70.01%-89.24% and end-to-end runtime down by 24.45%-55.97%.
  • The authors also report consistently improved task performance, though the size of the gain is not quantified in the text.
  • Code is public at github.com/THU-Agent/Coca-cola.

Why it matters

Agents that call tools tend to fill their context with raw outputs, which costs tokens and, per the authors, pulls attention away from what is relevant. Pruning and summarization still work in text, and the authors say that caps how much they can compress. This paper questions the premise that observations must be text at all, and reports much larger reductions (70.01%-89.24% in tokens) than a text-bound approach is said to allow.

Who it affects

Mainly people building LLM agents that read long tool outputs over multi-step interactions, such as web-search agents, the case the paper tests. Anyone paying for tokens or waiting on end-to-end runtime is the intended beneficiary, since the paper reports runtime cuts of 24.45%-55.97%.

How to use it

The authors link their code at https://github.com/THU-Agent/Coca-cola. The text gives no pricing, licence or setup details, so the repository is the place to check what is needed to try it. Conceptually, Coca-Cola sits between the tool and the agent: it takes the tool observation, produces latent tokens, and feeds those into the agent's input embedding space.

How solid is it

This is a preprint listed on Hugging Face Papers, and the evidence is the authors' own experiments: web-search agents, four QA benchmarks, two model families. The reported ranges are 70.01%-89.24% for token consumption and 24.45%-55.97% for end-to-end runtime, with consistently improved task performance. The text does not name the benchmarks or model families, does not say which end of each range belongs to which setting, and does not quantify the performance gain. The authors themselves word the conclusion modestly: the results suggest a promising approach.

Risks and caveats

Results are for web-search agents only; nothing is said about other agent types or tools. No baselines, such as pruning or summarization methods, are named or compared numerically in the text, so how much better Coca-Cola is than existing methods cannot be judged from it. The compression ratio per tool observation is not given, only overall ranges. No author names or institutions are given in the text. The name Coca-Cola has no stated connection to the beverage company.

“These results suggest that Coca-Cola offers a promising approach for context compression in agents.”

— From the paper's abstract