DiffusionGemma generates 1,500 tokens per second via parallel diffusion

The DiffusionGemma Team has introduced DiffusionGemma, an experimental open-weight language model built around discrete diffusion rather than the standard autoregressive (AR) approach most large language models use. Instead of predicting the next token one at a time, DiffusionGemma iteratively refines blocks of 256 tokens in parallel, which the authors say avoids the sequential decoding bottleneck that limits AR models. The model was not trained from scratch: it was obtained by fine-tuning the mixture-of-experts Gemma 4 model, which has 3.8B activated parameters and 25.2B total parameters. The fine-tuning pipeline is compute-efficient, using fewer than 10% of the total training token budget that went into the original AR model, and runs in two stages. The first stage applies supervised fine-tuning to teach the model bidirectional denoising. The second stage combines reinforcement learning with sampler distillation to improve both generation quality and inference efficiency at the same time. According to the authors, DiffusionGemma establishes a new Pareto frontier for the trade-off between generation speed and model capability. Averaged across their full evaluation suite, the model generates around 20 tokens per forward pass and reaches roughly 1,500 output tokens per second on a single NVIDIA H100 GPU, which the team describes as substantially faster than AR models even when those AR models use state-of-the-art speculative decoding. DiffusionGemma also retains the base model's support for thinking mode, multimodal inputs, and long contexts. Despite being fine-tuned for diffusion, the model reportedly remains capable of standard AR generation with only minor performance degradation, which the authors suggest points toward hybrid diffusion-AR decoding in the future. The technical report does not name individual authors beyond the collective team credit, does not give a publication or release date, does not cite the specific benchmarks or speculative-decoding baselines behind the speed comparison, and does not state a license or where the weights, despite being called open-weight, can be downloaded.

Key facts

DiffusionGemma is fine-tuned from the Gemma 4 mixture-of-experts model (3.8B activated, 25.2B total parameters) rather than trained from scratch.
The fine-tuning pipeline uses fewer than 10% of the original AR model's total training token budget, in two stages: supervised fine-tuning for bidirectional denoising, then reinforcement learning combined with sampler distillation.
The model refines blocks of 256 tokens in parallel and averages around 20 tokens generated per forward pass across the full evaluation suite.
Throughput reaches roughly 1,500 output tokens per second on a single NVIDIA H100 GPU, which the authors say beats AR models even against state-of-the-art speculative decoding.
DiffusionGemma keeps the base model's thinking mode, multimodal input support and long-context handling, and can still fall back to AR generation with only minor reported degradation.

Why it matters

Most large language models generate text one token at a time, which caps throughput no matter how much compute is thrown at the problem. DiffusionGemma reframes generation as parallel refinement of 256-token blocks through discrete diffusion, and the authors claim it establishes a new Pareto frontier between generation speed and model capability by reaching around 1,500 tokens per second on a single H100 while retaining most of the base model's abilities. If the approach generalizes, it points at a route to much lower latency without a corresponding drop in output quality.

Who it affects

The report speaks to researchers and engineering teams working on inference efficiency for large language models, particularly anyone weighing autoregressive speculative decoding against alternative decoding paradigms. Because DiffusionGemma keeps thinking mode, multimodal inputs and long-context support from its Gemma 4 base, it is positioned as a general-purpose model rather than a narrow speed demo, which also makes it relevant to teams evaluating latency-sensitive deployments.

How to use it

The technical report calls DiffusionGemma open-weight but does not state a license, a download location, or an availability timeline, so there is nothing here to point developers toward actually obtaining or deploying the model yet. The training recipe itself is efficient to reproduce in spirit: fine-tuning from an existing AR model using under 10% of that model's original training token budget, split between a supervised bidirectional-denoising stage and a reinforcement-learning-plus-sampler-distillation stage.

How solid is it

The results come from a single technical report published by the DiffusionGemma Team, with no individual authors named and no publication or release date given. The headline throughput and speed comparisons, including the claim of beating AR models against state-of-the-art speculative decoding, are the authors' own figures from their own evaluation suite; the report does not name the specific benchmarks or the speculative-decoding baselines being compared against, so there is no independent or third-party confirmation to check the numbers against.

Risks and caveats

Several details that would let outsiders verify or reproduce the claims are missing: no quantified figure for the "minor performance degradation" seen when the model falls back to AR generation, no description of the training data used in either fine-tuning stage, and no stated license or release plan despite the open-weight label. The reported 1,500 tokens-per-second figure is also specific to a single NVIDIA H100 GPU under the authors' own evaluation conditions, which may not carry over to other hardware or workloads.

“DiffusionGemma establishes a new Pareto frontier for the trade-off between generation speed and model capability.”

— the DiffusionGemma Team