DiffusionGemma averages about 1,500 tokens per second with parallel diffusion

Researchers have introduced DiffusionGemma, an experimental open-weight language model that uses discrete diffusion to generate text at high speed. Instead of decoding one token at a time like conventional autoregressive (AR) language models, it iteratively refines blocks of 256 tokens in parallel, which the authors say avoids the sequential decoding bottleneck of AR models.

DiffusionGemma was not trained from scratch. It comes from fine-tuning the mixture-of-experts Gemma 4 model, which has 3.8B activated parameters and 25.2B total parameters. The training pipeline runs in two stages and together uses fewer than 10% of the starting AR model's total training token budget. The first stage applies supervised fine-tuning to teach bidirectional denoising. The second stage combines reinforcement learning with sampler distillation to jointly improve generation quality and inference efficiency.

The authors say DiffusionGemma establishes a new Pareto frontier for the trade-off between generation speed and model capability. Averaged across their full evaluation suite, it generates around 20 tokens per forward pass and averages roughly 1,500 output tokens per second on a single NVIDIA H100 GPU, which they describe as substantially faster than AR models even when those models use state-of-the-art speculative decoding.

DiffusionGemma also keeps its starting model's support for thinking mode, multimodal inputs and long contexts. Despite the diffusion fine-tuning, it remains capable of ordinary autoregressive generation with only minor performance degradation, which the authors present as suggestive of a possible path toward hybrid diffusion-AR decoding. The technical report was submitted to arXiv on 31 July 2026.

Key facts

DiffusionGemma is an experimental open-weight language model that refines blocks of 256 tokens in parallel instead of decoding one token at a time.
It comes from fine-tuning the mixture-of-experts Gemma 4 model (3.8B activated parameters, 25.2B total parameters), not training a diffusion model from scratch.
Its two-stage training pipeline, supervised fine-tuning for bidirectional denoising followed by reinforcement learning combined with sampler distillation, uses fewer than 10% of the starting AR model's original training token budget.
Averaged across the authors' full evaluation suite, DiffusionGemma generates around 20 tokens per forward pass and averages roughly 1,500 output tokens per second on a single NVIDIA H100 GPU.
It keeps support for thinking mode, multimodal inputs and long contexts, and can still generate autoregressively with only minor, unquantified performance degradation.

Why it matters

Discrete diffusion is one of the more promising alternatives to the token-by-token decoding used by most mainstream large language models, because refining a whole block of tokens per step can sidestep the latency that sequential decoding imposes. What stands out here is not just the speed, roughly 1,500 tokens per second on a single NVIDIA H100 GPU, but how DiffusionGemma got there: rather than training a diffusion model from scratch, the authors fine-tuned an existing autoregressive model, the Gemma 4 mixture-of-experts model, using fewer than 10% of that model's original training token budget. The authors frame this as establishing a new Pareto frontier between generation speed and model capability, and they report that the fine-tuned model keeps most of its ability to fall back to ordinary AR generation, which they suggest could point toward models that combine both decoding modes.

Who it affects

Two groups are the natural audience for this report. Teams running large language models where response latency matters, such as chat assistants, coding agents or high-volume APIs, stand to gain from a model that averages roughly 1,500 tokens per second on a single H100 GPU, since that speed could cut serving time and cost without dropping down to a smaller, weaker model. Researchers working on diffusion-based language generation are the other audience: DiffusionGemma is presented as evidence that an existing autoregressive model can be turned into a fast diffusion model through comparatively cheap fine-tuning, rather than needing to train a diffusion model from the ground up.

How to use it

DiffusionGemma is described as open-weight, but the report gives no release date, download location or license terms, so that claim comes with no timeline attached in this technical report. For now, the only way to engage with the work is to read the report itself, posted to arXiv as paper 2608.00146 on 31 July 2026.

How solid is it

This is a single technical report posted to arXiv, not a peer-reviewed publication, and the source text does not name the paper's authors; the only individual named is an arXiv submission contact, not identified as an author. The two headline performance figures, around 20 tokens per forward pass and roughly 1,500 tokens per second, are both averages across what the report calls its full evaluation suite, with no individual benchmark names, per-task scores or error bars given for either number. The claim that DiffusionGemma is substantially faster than AR models even with state-of-the-art speculative decoding does not name which AR models were used for the comparison, so the size of the advantage cannot be checked against a specific competing system from this report alone.

Risks and caveats

The minor performance degradation the authors report when DiffusionGemma is used for ordinary autoregressive generation is not given a number, only described as minor, so the practical cost of falling back to AR mode is not quantified here. The headline speed figures are averages across the full evaluation suite rather than a per-task breakdown, so performance on any single kind of task could differ from the averages. And because no release date or venue is given for the weights, none of these figures can currently be independently reproduced by outside developers running the model themselves.