Glonce

Paper finds LLMs use fewer tokens per task when tasks are combined in one prompt

researchThe Glonce editorsHugging Face🔥60Glonce rating60Impact40Novelty60Relevance95Surprise50Players20Credibility65
Paper finds LLMs use fewer tokens per task when tasks are combined in one prompt

A paper listed on Hugging Face Papers, titled "On the Fragility of Tasks under Composition: Implicit Budgeting in Language Models", looks at what happens when a language model is asked to handle several tasks within a single prompt. That is an increasingly common use. It requires solving each task correctly and then integrating the answers into one coherent response.

The central finding is that models make substantially more errors when tasks are composed, even when they can solve every constituent task reliably in isolation. The authors say this degradation cannot be fully explained by the accumulation of independent errors.

Their explanation is an implicit thinking budget. Across both open-weight models and the frontier model Claude Opus 5, the findings point to this effect: under composition, models use substantially fewer tokens per task than they do in isolation. This happens even when the full sequence remains well within the context window, so running out of room is not the cause.

A consequence follows. Tasks that need longer solution traces in isolation are more fragile under composition, meaning models make more errors on prompts that contain them. The reverse also holds: in isolation, non-fragile tasks receive more reasoning than they need and waste compute.

That last point gives the authors a practical lever. By grouping tasks according to their fragility, they say the implicit budgeting can be manipulated to reduce token usage by 43% across a range of math and linguistic tasks, while also preserving accuracy.

Key facts

  • Models make substantially more errors when tasks are composed in one prompt, even if each task is solved reliably in isolation.
  • The authors say independent error accumulation cannot fully explain the drop; they point to an implicit thinking budget.
  • Under composition, models use substantially fewer tokens per task, even when the sequence fits well inside the context window.
  • Tasks that need longer solution traces in isolation are more fragile when composed.
  • Grouping tasks by fragility cut token usage by 43% across math and linguistic tasks with accuracy preserved, per the authors.

Why it matters

Putting several tasks into one prompt is an increasingly common way to use language models. The paper identifies what the authors call a fundamental limitation of that practice: reliability on each task alone does not carry over to the combined prompt. The explanation they offer is not a lack of context room but an implicit thinking budget, so models spend fewer tokens on each task when there are more of them.

Who it affects

Anyone who bundles multiple questions or subtasks into one request to a language model. The effect was reported for open-weight models and for the frontier model Claude Opus 5. It also bears on people who care about compute cost, since the authors say non-fragile tasks get more reasoning than they need when run alone.

How to use it

The authors' practical suggestion is to group tasks according to their fragility, which they say reduces token usage by 43% across a range of math and linguistic tasks while preserving accuracy. Tasks that need long solution traces in isolation are the fragile ones, so they are the ones at risk when composed. The abstract does not describe a step-by-step procedure.

How solid is it

This rests on the paper's abstract as shown on Hugging Face Papers. The claims are the authors' own findings across open-weight models and Claude Opus 5. The 43% figure is stated without saying whether it is a mean, median or maximum, and without a baseline token count. The abstract does not name the open-weight models, list the tasks or benchmarks, give baseline error rates, or say how many tasks were composed per prompt.

Risks and caveats

The authors argue the effect cannot be fully explained by independent errors and point to an implicit thinking budget, but the abstract gives no mechanism for why that budget arises. The 43% saving is reported for math and linguistic tasks, and nothing here says it holds elsewhere. Without the size of the error increase, the scale of the fragility problem is unclear beyond 'substantially more'.