Paper says training speculative-decoding drafts to be rejected helps long-horizon agents

The paper starts from a common assumption: speculative decoding is a systems optimization that speeds up large language model decoding without changing model behavior, and the effort to build better draft models has focused solely on acceptance rate. The authors argue this view does not hold for long-horizon agent reasoning.

Their evidence comes from measuring frontier open-weight models on complex agentic tasks with different speculative decoding approaches. They report that numerical noise grows within verification windows. They also report that the draft acceptance results of the thinking process can change the model's output distribution after 30 steps, which noticeably influences the reasoning accuracy of long-horizon agents in random directions.

The fix is to control that noise. The authors say they cannot change the overall token distribution verified by the target model. What they can do is encode critical reasoning with the highest precision by training the draft model to be rejected. They identify critical decision points in agent traces and train the draft model to favor an adversarial preference. At inference, the trained draft and the target part ways at exactly those points and verification restarts. Critical thinking tokens are then sampled from the target model and encoded under its least noisy conditions. The authors say this makes critical reasoning better indexed and more durable over a long horizon.

On complex agentic coding benchmarks, the approach raises end-to-end success rate with no change to the target model or its output distribution. The authors say the result recasts speculative decoding from a pure accelerator into an active allocator for context precision. The stated cost is training 0.06% to 0.28% of hosted parameters.

Key facts

The authors challenge the view that speculative decoding is purely a speed optimization, arguing it does not hold for long-horizon agent reasoning.
Measuring frontier open-weight models on complex agentic tasks, they found numerical noise grows within verification windows, and draft acceptance can change the output distribution after 30 steps.
The method identifies critical points in agent traces and trains the draft model to be rejected there, so the target model samples those tokens under its least noisy conditions.
On complex agentic coding benchmarks it raises end-to-end success rate with no change to the target model or its output distribution.
The stated training cost is 0.06% to 0.28% of hosted parameters.

Why it matters

Draft-model training has been judged mostly by acceptance rate, meaning how often the target model agrees with the draft. This paper claims that for long agent runs, acceptance results can themselves shift the model's output distribution after 30 steps and nudge reasoning accuracy in random directions. If that holds, a faster decoding setup is not automatically a neutral one. The authors' answer is to use the draft model to decide where the target model should write the tokens itself, treating speculative decoding as a way to allocate precision rather than only speed.

Who it affects

The work is aimed at people who run long-horizon agents, especially coding agents, on top of speculative decoding, and at those who train draft models. The measurements were made on frontier open-weight models, so teams serving that class of model are the closest audience.

How to use it

The abstract describes a recipe, not a release. You would identify critical decision points in agent traces, train the draft model to favor an adversarial preference at those points, and then run inference as usual, with verification restarting wherever the draft and target diverge. The paper states the training touches 0.06% to 0.28% of hosted parameters. No code, release or availability information is given.

How solid is it

This is a preprint summarized from its abstract on a Hugging Face papers page. The claims are the authors' own. No authors or institutions are named. No benchmark names, models tested, or success-rate figures (baseline or improved) are given; the size of the end-to-end improvement is not stated. The source also does not say which frontier open-weight models were measured. Treat the result as a claim to be checked against the full paper.

Risks and caveats

No speedup or latency numbers are given, so the effect of forcing rejections on decoding speed is unknown from this text. The 0.06% to 0.28% range is not tied to specific models or draft sizes in the abstract. The 30-step threshold and the noise findings come from the authors' own measurements. The improvement is reported only for agentic coding benchmarks.

“While we cannot change the overall token distribution verified by the target model, we can encode critical reasoning with highest precision by training the draft model to be rejected.”

— From the paper's abstract