TraceQuery cuts fresh computation in diffusion LLM decoding by recomputing only trajectory-relevant tokens
Diffusion large language models (dLLMs) generate text through iterative masked denoising. That flexibility has a price: inference is expensive, because token representations are recomputed again and again across denoising steps.
Existing cache-based methods cut this cost by reusing intermediate results from earlier steps and recomputing only a subset of positions at each step. Which positions get recomputed is decided either on a fixed schedule or by heuristic signals of change. The authors of this paper argue from their trajectory experiments that a different quantity matters more: the decoding trajectory.
Their central finding is that cached generation stays accurate when it remains close to the full-denoising trajectory. Errors concentrate around deviations in the unmask order, and they shrink as longer reference prefixes are recovered. From this the authors draw a principle: limiting representation error helps, but keeping generation on the full-denoising trajectory is a more direct route to accurate decoding.
On that basis they reframe efficient dLLM inference as trajectory-guided querying. Instead of broadly repairing representation error, they use full-denoising trajectories to supervise which unresolved positions should receive a limited amount of fresh computation.
The resulting method is TRACEQUERY, described as a lightweight sparse decoding framework. At each step it predicts a small set of near-future trajectory-relevant positions and queries only those candidates. The backbone model then picks which one to commit from the fresh predictions.
The authors say TRACEQUERY delivers the strongest quality-computation trade-off among cache-based approaches. They report that it sustains high accuracy with substantially fewer fresh token computations, and that it generalizes across tasks, model variants and families. The abstract gives no figures, benchmarks or model names. Code is released at https://github.com/fish1928/tracequery.git.
Key facts
- TRACEQUERY is a lightweight sparse decoding framework for diffusion LLMs that queries only a small set of predicted near-future, trajectory-relevant positions.
- The authors find cached generation stays accurate when it stays close to the full-denoising trajectory; errors cluster around unmask-order deviations and fall as longer reference prefixes are recovered.
- Full-denoising trajectories are used as supervision to decide which unresolved positions get limited fresh computation, replacing fixed schedules and heuristic change signals.
- The backbone model selects which queried candidate to commit, based on its fresh predictions.
- The authors claim the best quality-computation trade-off among cache-based approaches and generalization across tasks, model variants and families; no numbers are given in the abstract, and code is on GitHub.
Why it matters
Diffusion language models are an alternative to left-to-right generation, but they pay for their flexibility by recomputing token representations at every denoising step. Cache-based methods try to reduce that bill. This paper argues the field has been choosing what to recompute by the wrong criterion, such as a fixed schedule or heuristic signs of change, when the decoding trajectory is the more relevant quantity. If that holds, it gives cache design a clearer target: stay on the full-denoising path rather than minimize representation error broadly.
Who it affects
The work is aimed at researchers and engineers building or serving diffusion LLMs, especially anyone already using cache-based decoding to reduce inference cost. The abstract does not name specific models or deployments.
How to use it
The authors have released code at https://github.com/fish1928/tracequery.git, so practitioners working with dLLM decoding can try the method directly. In outline, a lightweight component predicts a few near-future trajectory-relevant positions, only those are recomputed, and the backbone decides which one to commit. The abstract does not give training details or the size of the position predictor, so those need to be checked in the paper and repository.
How solid is it
This is an arXiv preprint, and the claims come from the authors' own abstract. The strongest claim, the best quality-computation trade-off among cache-based approaches, is stated without figures, named benchmarks or named baselines in the text available here. Code is public, which allows independent checking. Generalization across tasks, model variants and families is asserted rather than shown in the abstract.
Risks and caveats
The comparison is among cache-based approaches; the source does not claim wall-clock speedups over full denoising. With no numeric results in the abstract, the size of the saving in fresh token computations is unknown to the reader. The finding rests on the authors' trajectory experiments, and results may vary outside the tasks and models they tested.