HDM hybrid model decides when to answer without writing reasoning tokens

HDM hybrid model decides when to answer without writing reasoning tokens

The paper starts from a complaint about adaptive-reasoning methods. Existing ones either shorten the explicit reasoning a model writes, or skip it by switching modes. Either way, the answer is still written out token by token. The authors set aside the question of how much to think and propose a different paradigm, which they call zero-or-T thinking. A single model either answers from the implicit thinking inside one forward pass, generating no token, or performs full autoregressive reasoning at a cost of T tokens, when that is worth paying.

They build this as a hybrid decision model (HDM). One frozen backbone serves both modes, and a lightweight self-routing block chooses between them. The block estimates the value of thinking, meaning the utility the slow mode is expected to add over the fast one. It does this from what the implicit thinking has already produced. The slow mode runs only where that value exceeds a price λ set per request.

Some tasks, such as episodic agentic ones, have no per-decision label to train the router on. For those, the authors introduce Calibrated Decision Policy Optimisation (CDPO), a reinforcement-learning recipe that trains the block from episode-level returns.

HDM is evaluated on six benchmarks: MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77 and MiniWoB. Two results are given in numbers. On JevBench, HDM-4B scores 0.857 against Jev's 0.851 (+0.7%), at a median 0.179 s against 0.652 (-72.5%). On the agentic benchmark MiniWoB, HDM-27B cuts token cost by 80.1%.

Key facts

  • Zero-or-T thinking: one model either answers inside a single forward pass with no generated tokens, or does full autoregressive reasoning costing T tokens.
  • The hybrid decision model (HDM) uses one frozen backbone for both modes and a lightweight self-routing block that picks between them.
  • The router estimates the value of thinking from the implicit thinking already produced, and runs the slow mode only where that value exceeds a price λ set per request.
  • For agentic tasks without per-decision labels, Calibrated Decision Policy Optimisation (CDPO) trains the router from episode-level returns with reinforcement learning.
  • Reported results: on JevBench HDM-4B scores 0.857 against Jev's 0.851 (+0.7%) at a median 0.179 s against 0.652 (-72.5%); on MiniWoB HDM-27B cuts token cost by 80.1%.

Why it matters

Most adaptive-reasoning work trims how many reasoning tokens a model writes. This paper changes the choice: for each request, write no tokens at all or reason in full. The authors say existing methods still write the answer out token by token, and that is the cost they aim to avoid. The reported JevBench numbers show a small score gain (0.857 against 0.851) together with a 72.5% cut in median time. The MiniWoB result is an 80.1% cut in token cost for HDM-27B on an agentic benchmark.

Who it affects

The work is aimed at people who run reasoning models and pay for generated tokens or wait on latency, including agentic setups where episodes are long. It also touches researchers working on adaptive reasoning, since the paper proposes a routing design and a training recipe (CDPO) for cases with no per-decision labels.

How to use it

The abstract describes a method, not a product. The practical lever it describes is the price λ, which is set per request: the slow mode runs only where the estimated value of thinking exceeds it. No code, model release, training cost or limitations are mentioned.

How solid is it

This rests on a paper abstract that reports results on six benchmarks: MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77 and MiniWoB. Only two results are given as numbers, the JevBench comparison for HDM-4B and the MiniWoB token-cost cut for HDM-27B. It does not say what 'Jev' is (a model, a method or a baseline), nor which metric the 0.857 and 0.851 figures are. The baseline for the 80.1% token-cost reduction is not named. The headline gain on JevBench is small (+0.7%); the larger effect is on time and token cost. The abstract names no authors or institutions.

Risks and caveats

No results are given for MMLU-Pro, AGIEval, GPQA-Diamond or BANKING77, and none for HDM-27B beyond the MiniWoB token-cost figure. The value of T, the sizes or identity of the frozen backbone, and the value or typical range of λ are not stated. The unit of the 0.652 comparison figure is not restated; only 0.179 s carries its unit. Without the full paper, the size of the effect on other benchmarks and the cost of routing mistakes are unknown.

“a single model either answers from the implicit thinking inside one forward pass, generating no token, or performs full autoregressive reasoning at a cost of T tokens when that is worth paying”

— Abstract of the paper "The Value of Thinking: Learning When to Reason in Hybrid Decision Models"