Glonce

D3D diffusion drafter reaches 5.00x speedup over DFlash's 4.03x on Qwen3-8B

researchThe Glonce editorsHugging Face🔥62Glonce rating62Impact50Novelty60Relevance100Surprise35Players15Credibility70
D3D diffusion drafter reaches 5.00x speedup over DFlash's 4.03x on Qwen3-8B

Speculative decoding uses a small drafter to propose candidate tokens that a larger target model then verifies. The paper starts from a bottleneck in one family of drafters. Target-conditioned diffusion drafters predict candidate tokens in parallel, but each new block has to wait for updated hidden states from the target model. That forces drafting and verification to run one after the other, which limits the speedup even though every block is produced in a single pass.

The authors introduce D3D, a standalone diffusion drafter that removes this dependency between rounds. D3D combines blockwise denoising with auxiliary autoregressive supervision, which the authors say lets it achieve long accepted prefixes without using the target model's hidden states. Because it does not depend on those states, D3D can draft successor blocks while verification is still running, in one packed pass. This is what the paper calls asynchronous speculative decoding.

Packed drafting has a cost, so the authors also prune the drafter. They remove layers progressively, ranked by how much each one affects acceptance length. Removing five of the drafter's 36 layers retains over 99% of mean acceptance length across three benchmarks, with Qwen3-8B as the target.

On speed, the authors report results across seven benchmarks. D3D averages 5.00x decoding speedup with a Qwen3-8B target and 5.71x with Qwen3-32B. The DFlash comparison method averages 4.03x and 3.67x on the same two targets. The best single result given is on GSM8K with Qwen3-32B, where speedup reaches 7.48x. Code is publicly available at https://github.com/fudan-generative-vision/D3D.

Key facts

  • D3D is a standalone diffusion drafter that does not need target-model hidden states, so drafting and verification no longer have to run in sequence.
  • It can draft successor blocks during verification in one packed pass, which the authors call asynchronous speculative decoding.
  • Averaged over seven benchmarks, D3D reaches 5.00x (Qwen3-8B) and 5.71x (Qwen3-32B) decoding speedup, versus 4.03x and 3.67x for DFlash.
  • On GSM8K with Qwen3-32B, the speedup reaches 7.48x.
  • Removing five of 36 layers keeps over 99% of mean acceptance length across three benchmarks with Qwen3-8B as the target.

Why it matters

Diffusion drafters generate a whole block of candidate tokens in one pass, but the authors say the need for fresh target hidden states for every new block forces drafting and verification to take turns. D3D is built to break that wait. By not depending on target hidden states, it can work on the next blocks while the target is still verifying, and the reported gain over DFlash is largest on the bigger Qwen3-32B target (5.71x against 3.67x on average).

Who it affects

The work is aimed at people who serve or study large language models and care about decoding speed, especially those using speculative decoding with diffusion-style drafters. The evaluation uses Qwen3-8B and Qwen3-32B as targets.

How to use it

Code is publicly available at https://github.com/fudan-generative-vision/D3D. The abstract describes the approach as a standalone drafter with layers pruned by their effect on acceptance length, but it gives no setup instructions, hardware requirements or licence terms.

How solid is it

The numbers are the authors' own, averaged over seven benchmarks, and the comparison is against DFlash on two Qwen3 targets. The pruning result (over 99% of mean acceptance length after removing five of 36 layers) is reported only for the Qwen3-8B target and across three benchmarks. The abstract does not list the benchmarks other than GSM8K. This is a preprint summary, so there is no independent replication to point to.

Risks and caveats

The abstract does not state what the speedup is measured against, nor the hardware, batch size or memory use behind it, so the figures may not carry over to other setups. Absolute acceptance-length values are not given, and no pruning figure is given for Qwen3-32B. The 7.48x result is a single benchmark and target, not an average.