Gambit brings thought-level beam search to reasoning models

Large reasoning models get much of their performance today from test-time compute scaling, but current approaches spend that compute inefficiently, which shifts the real question from how much compute to use to where to allocate it. A new paper formalizes test-time reasoning as a constrained compute allocation problem over partial reasoning trajectories. The authors write that under a fixed hardware budget, existing paradigms fail to actively route compute to the most promising partial progress: traditional parallel sampling treats reasoning traces independently and induces severe memory bottlenecks, while subtractive pruning starves the hardware and fails to actively and sufficiently shift the output distribution toward better answers.

To close that gap, the authors introduce Gambit, an inference algorithm that executes thought-level beam search. Gambit periodically prunes unpromising trajectories and immediately branches from high-quality prefixes, using a lightweight scorer that probes the model's hidden states to judge which partial traces are worth continuing. This lets it dynamically concentrate compute on the most promising reasoning traces while maintaining continuous high hardware utilization, instead of letting traces sit idle or compete independently for memory.

In evaluations across multiple models and benchmarks, the authors write that Gambit strictly dominates existing baselines. Under identical hardware constraints, it yields up to a 6.7% absolute accuracy gain on the HMMT-24 benchmark and up to a 3.3% absolute gain on AIME-25, both measured over pruning baselines. It also delivers more than double (>2x) the throughput on trace completion, and it reduces total token consumption by up to 68.5% relative to standard parallel sampling.

Key facts

Gambit formalizes test-time reasoning as a constrained compute allocation problem, addressing where to spend test-time compute rather than just how much to spend.
It runs thought-level beam search: periodically pruning unpromising reasoning trajectories and branching from high-quality prefixes, guided by a lightweight scorer that probes the model's hidden states.
Under identical hardware constraints, Gambit gains up to a 6.7% absolute accuracy improvement over pruning baselines on HMMT-24 and up to 3.3% on AIME-25.
Gambit delivers more than double (>2x) the throughput on completing reasoning traces under the same hardware constraints.
It cuts total token consumption by up to 68.5% relative to standard parallel sampling.

Why it matters

Test-time compute scaling is now one of the main levers for improving large reasoning models, but wasting that compute limits how much a model can gain from it. The paper reframes the problem: instead of asking how much compute to spend, it asks where to allocate it across partial reasoning trajectories. The authors say that reframing targets a real inefficiency: parallel sampling explores many traces independently and runs into severe memory bottlenecks, while pruning approaches save memory but starve the hardware and do not sufficiently shift results toward better answers. Gambit's thought-level beam search is presented as a way to get both memory efficiency and continuous high hardware utilization at the same time.

Who it affects

This is an inference-time technique, not a training method, so it mainly matters to teams that build or serve large reasoning models under a fixed hardware or compute budget: model providers and inference infrastructure engineers looking to raise accuracy or cut cost per query without retraining. Indirectly, it affects anyone using a reasoning model that adopts a technique like this, since faster trace completion and lower token consumption can translate into cheaper or faster answers.

How to use it

Gambit operates at inference time: it periodically prunes unpromising partial reasoning traces and immediately branches from the strongest prefixes, guided by a lightweight scorer that probes the model's hidden states to judge which traces to keep. The source does not mention whether code, the trained scorer, or an evaluation harness will be released, so for now it is described as a method rather than published as a ready-to-use tool.

How solid is it

The authors report evaluations across multiple models and benchmarks and write that Gambit strictly dominates existing baselines, with all comparisons run under identical hardware constraints, which controls for a major confound in compute-efficiency claims. The specific figures given are tied to two named benchmarks, HMMT-24 and AIME-25, plus separate throughput and token-consumption comparisons; the source does not specify which additional models or benchmarks were covered by 'multiple models and benchmarks,' and it gives no information on peer review, publication venue, or citation count, so the claims rest on the authors' own reported evaluations.

Risks and caveats

The headline numbers are framed as ceilings, not averages: 'up to' a 6.7% accuracy gain on HMMT-24 and 'up to' 68.5% token savings, so actual gains vary and can be smaller under some conditions. The throughput claim, more than double on trace completion, does not name a comparison baseline in that part of the text. The mechanism of the lightweight scorer that probes hidden states is not explained in architectural or training detail. The paper does not mention code, model, or dataset release.