New Uno models cut LLM inference latency 3x with no quality loss

A new paper proposes diffusion-augmented LLMs, a class of models named Uno that keeps the standard autoregressive, next-token-prediction distribution of a language model but uses diffusion to draw several tokens from that distribution in parallel, rather than one at a time. The approach splits the model's parameters into two sets: the original autoregressive weights, trained the normal way with next-token prediction, and a separate, lightweight set of diffusion weights whose job is to generate multiple tokens simultaneously. The diffusion weights are trained through what the authors call a Diffusion Distillation phase, which they say adds negligible overhead on top of an existing LLM training pipeline. Alongside Uno, the paper introduces Psi-Spec, a family of samplers that the authors say enables lossless acceleration and lets inference scale up at a fixed context length. The authors distinguish the method from two existing approaches. Unlike speculative decoding, Uno needs no separate draft model to propose candidate tokens. Unlike diffusion LLMs (d-LLMs), which generate text non-autoregressively throughout, Uno keeps the underlying autoregressive model's output distribution intact, so the authors describe the speedup as lossless rather than a quality tradeoff. Uno models can either be trained from scratch or built by adding the diffusion weights on top of an existing open-weight autoregressive LLM. On the results the authors report, Uno achieves higher throughput than the leading speculative-decoding methods at every batch size they evaluated, and delivers speedups of up to 3x over the base autoregressive model, including at the largest batch size the test device supported. They also report that an 8B-parameter Uno model outperforms two larger or closed alternatives, the 26B open-weight DiffusionGemma (described as the leading open d-LLM) and the proprietary Mercury 2, across all the benchmarks they evaluated in agentic tool use, coding, and long-context reasoning. The text does not specify which benchmarks were run, give raw scores, or state absolute latency, cost, or hardware figures behind the throughput comparisons; it also gives no detail on training data or compute cost beyond calling the distillation phase lightweight. The authors say they are releasing code and checkpoints at a linked project site, though the text gives no license terms or release date.

Key facts

Uno pairs a standard autoregressive LLM with separate, lightweight diffusion weights that generate multiple tokens in parallel from the same output distribution.
The diffusion weights are trained via a Diffusion Distillation phase that the authors say adds negligible overhead to existing training pipelines.
Uno needs no separate draft model, unlike speculative decoding, and preserves the base model's output quality, unlike existing diffusion LLMs.
Uno delivers up to 3x speedups over the base autoregressive model and beats leading speculative-decoding methods at every evaluated batch size.
An 8B Uno model reportedly outperforms the 26B open DiffusionGemma and the proprietary Mercury 2 across all evaluated benchmarks in agentic tool use, coding, and long-context reasoning.

Why it matters

Autoregressive generation, the standard way LLMs produce text, is inherently sequential: one token at a time, each waiting on the last. That is the main reason inference is slow and expensive at scale. Uno's pitch is that this bottleneck can be loosened without touching the underlying model's output distribution, by bolting on a small diffusion component that samples several tokens per step. If the reported gains hold up outside the paper's own benchmarks, it is a way to make existing autoregressive LLMs measurably cheaper to run rather than a new way to train them from zero.

Who it affects

The immediate audience is teams that train or serve large language models and care about inference cost and latency: model providers running high-volume generation, and researchers working on decoding methods such as speculative decoding or diffusion LLMs, which the paper explicitly positions itself against. Because Uno can be built on top of an existing open-weight autoregressive model rather than requiring training from scratch, it is also relevant to anyone maintaining an already-deployed open model who wants faster inference without retraining it from the ground up.

How to use it

The authors say they are releasing code and checkpoints at a linked project site, and describe Uno as buildable either from scratch or by augmenting an existing open-weight autoregressive LLM through the Diffusion Distillation phase. The text does not give a license, a release date, which base models are supported, or any pricing or hosting terms, so none of that can be stated beyond the release itself.

How solid is it

The claims come from the paper's own abstract and are attributed to the authors throughout; no independent benchmarking is described. The headline results, higher throughput than leading speculative-decoding methods at every evaluated batch size and up to 3x speedup over the base model, are the authors' own reported figures, as are the comparisons showing an 8B Uno model beating the larger 26B DiffusionGemma and the proprietary Mercury 2. The abstract names no authors or institutions, gives no benchmark names or raw scores, and states no absolute latency, cost, or hardware specification for the device used in testing.

Risks and caveats

The comparisons are described only in relative terms: 'up to 3x' speedup and 'outperforms across all evaluated benchmarks' with no scores, benchmark names, or hardware details given, which makes the results hard to independently verify or reproduce from the text alone. The training data, compute cost, and the actual size of the Diffusion Distillation phase are not disclosed beyond it being called lightweight and low-overhead. No license or release date accompanies the promised code and checkpoints.