Glonce

ReaDy speeds up Qwen3.8-27B decoding 3.16x with recurrent-state drafting

researchThe Glonce editorsHugging Face🔥56Glonce rating56Impact42Novelty58Relevance100Surprise18Players8Credibility72
ReaDy speeds up Qwen3.8-27B decoding 3.16x with recurrent-state drafting

Speculative decoding speeds up text generation by drafting several tokens per step and letting the main model check them. The paper behind ReaDy starts from a weakness it sees in feature-based drafters: they depend on the target model's hidden features, and the context those features emphasise may not match what later draft positions need.

The authors' answer uses hybrid linear attention (HLA) models, which keep compact recurrent states that accumulate historical associations. They argue these states are an efficient source of complementary context for drafting. ReaDy, which they call the first speculative decoding method, to the best of their knowledge, to use HLA recurrent states for position-specific context retrieval, gives each draft position a learned query. That query pulls supplementary context straight from the target model's committed recurrent states. A lightweight state reader then folds this context into the draft representations through residual updates. Low-rank state reads and fused operators are meant to keep the extra computation and I/O overhead small.

ReaDy builds on DSpark's parallel backbone and sequential prediction head and uses a lightweight two-stage adaptation strategy. The authors examine reader capacity, state sources and residual placement through ablation studies.

On results, the paper reports that across seven benchmarks spanning mathematics, code generation and dialogue, ReaDy reaches an average 3.16x decoding speedup over autoregressive decoding with Qwen3.8-27B on a single NVIDIA B200 GPU. Against DSpark, it wins on every benchmark, improving throughput by 7.97% and acceptance length by 8.50% on average. The code is published on GitHub under the StartLuxLabs organization.

Key facts

  • ReaDy lets each draft position use a learned query to retrieve context from the target model's committed recurrent states in hybrid linear attention (HLA) models.
  • Reported average decoding speedup is 3.16x over autoregressive decoding, across seven benchmarks (mathematics, code generation, dialogue), with Qwen3.8-27B on a single NVIDIA B200 GPU.
  • Against the DSpark baseline, ReaDy is better on every benchmark, with average gains of 7.97% in throughput and 8.50% in acceptance length.
  • A lightweight state reader adds context through residual updates; low-rank state reads and fused operators are used to limit extra computation and I/O.
  • Code is available at github.com/StartLuxLabs/ReaDy.

Why it matters

Speculative decoding is a way to make generation faster without changing what the target model produces, and drafters are only as useful as the context they can see. ReaDy's idea is to tap a resource that hybrid linear attention models already carry, their compact recurrent states, instead of relying only on hidden features. The authors say it is the first method to do this for position-specific retrieval during drafting, as far as they know. If the reported gains hold, it points to a cheap source of extra drafting signal for this model family.

Who it affects

The work is aimed at people building or serving hybrid linear attention models who want faster decoding, and at researchers working on speculative decoding and drafter design. The reported measurements involve Qwen3.8-27B, so teams using that model are the closest match. Anyone already using DSpark is the direct comparison point, since ReaDy builds on its backbone and prediction head.

How to use it

The authors link code at https://github.com/StartLuxLabs/ReaDy. The method is described as a lightweight two-stage adaptation of a DSpark-style drafter, with a state reader added on top. The abstract gives no price or licence terms. Anyone trying it would want to look at the ablations on reader capacity, state sources and residual placement, which the authors say they ran.

How solid is it

The claims come from the paper's abstract, and the speedup and gains are the authors' own measurements. The headline figures are averages: 3.16x over autoregressive decoding, 7.97% throughput and 8.50% acceptance length over DSpark, and the authors say ReaDy beats DSpark on every one of seven benchmarks. The abstract does not give per-benchmark results, and it does not say which benchmarks were used. It also does not say whether the 7.97% and 8.50% gains are relative or absolute points, nor the DSpark baseline values. Independent replication is not mentioned.

Risks and caveats

The 3.16x speedup is stated only for Qwen3.8-27B on one B200 GPU; no other models or hardware are reported, so it is unclear how far the result carries over. No training cost, memory overhead or latency figures for the state reader are given, and the abstract does not state the batch size or decoding settings used. The approach depends on HLA recurrent states, so it applies to hybrid linear attention targets rather than any model. The claim of being first is hedged by the authors themselves with 'to the best of our knowledge'.

“ReaDy equips each draft position with a learned query to retrieve supplementary context directly from the target's committed recurrent states.”

— ReaDy paper abstract