Glonce

Liquid AI ships DSpark draft models for 3.2x faster LFM2.5 inference

infrastructureThe Glonce editorsHugging Face🔥63Glonce rating63Impact60Novelty50Relevance92Surprise40Players38Credibility88
Liquid AI ships DSpark draft models for 3.2x faster LFM2.5 inference

Liquid AI released DSpark, a set of speculative-decoding draft models built for its LFM2.5 model family, reporting up to 3.18x throughput improvement on a GPU and up to 2.87x on-device, and open-sourcing the LFM-compatible integration upstream in llama.cpp and SGLang from day one.

DSpark targets the decode phase of LLM inference, which is typically memory-bound: most latency comes from streaming weights from DRAM into SRAM rather than from computation itself. Speculative decoding addresses this by having a small draft model propose candidate tokens, which the larger target model then verifies in a single forward pass, spreading the cost of loading weights across every verified token. DSpark combines three components: a DFlash-style parallel backbone that produces hidden states for all draft tokens in one forward pass conditioned on the target model's context features; a lightweight sequential head, modeled as a Markov chain between neighboring tokens, that raises the acceptance rate at later positions; and a confidence-scheduled verifier that predicts each token's survival probability and prunes low-confidence suffixes when verification would cost more than it saves.

Liquid AI trained the draft models on a larger, more diverse data mix covering SFT, chat, code and function-calling data, following the DSpark recipe. The first-version draft models are simplified attention-only architectures with 5 layers and a block size of 9; each was trained for 15 epochs, with the epoch chosen by highest acceptance rate rather than lowest loss. Each resulting draft model has about 300 million parameters. Because a draft token is accepted only when it matches the target model's own distribution under greedy decoding, and the target's token replaces any rejected draft token, the emitted sequence is identical to baseline greedy decoding by construction, so benchmark accuracy such as pass@1 or exact match is unchanged.

Liquid AI measured on-device throughput with llama.cpp and Metal on an M4 Max MacBook Pro using FP16 GGUF weights and up to 256 output tokens, and GPU throughput with SGLang on a single H100 80GB card in BF16, across five benchmark datasets, all with a DSpark block size of 9, batch size of 1 and temperature of 0. For LFM2.5-2.6B, the MacBook speedup is described as especially noticeable, pushing interactivity beyond the roughly 140 tokens per second Liquid AI attributes to most proprietary cloud models, and DSpark cuts function-calling latency by 57% on average across multi-tool scenarios for that model. For LFM2.5-1.2B-Instruct, acceptance rates vary more by dataset, so speedup varies by as much as 52% depending on the text distribution. For LFM2.5-8B-A1B, the acceptance rate is higher than for two dense models, yet the on-device improvement averages only 18%; Liquid AI attributes this gap to the current MoE implementation in llama.cpp's Metal backend and to the fact that verifying multiple tokens activates more experts, and therefore more weight traffic, than a single decode step.

To run DSpark with SGLang, Liquid AI says an SGLang build with DSpark support for LFM2 targets is required (tracked in PR #31041); the target model is launched with the draft model attached via a set of speculative-decoding flags, and queried through the standard OpenAI-compatible endpoint. Running DSpark with llama.cpp requires the corresponding llama.cpp build (PR #27383) and a set of spec-type flags pointing at the draft model. In both cases the block size is read automatically from the draft model's own configuration. The DSpark draft-model checkpoints for LFM2.5-2.6B, LFM2.5-1.2B-Instruct and LFM2.5-8B-A1B are published on Hugging Face in both Safetensors and GGUF format.

Key facts

  • DSpark, Liquid AI's speculative-decoding draft models for LFM2.5, delivers up to 3.18x GPU throughput and up to 2.87x on-device throughput improvement.
  • For LFM2.5-2.6B, DSpark cuts function-calling latency by 57% on average across multi-tool scenarios.
  • Speedup varies by model: as much as 52% variance for LFM2.5-1.2B-Instruct depending on the dataset, and only an 18% average on-device gain for the MoE model LFM2.5-8B-A1B, attributed to llama.cpp's Metal MoE implementation.
  • Because rejected draft tokens are replaced by the target model's own output under greedy decoding, DSpark output is identical to baseline greedy decoding, leaving benchmark accuracy unchanged.
  • The draft models, each about 300M parameters, ship with day-one support in llama.cpp and SGLang and are published on Hugging Face in Safetensors and GGUF formats.

Why it matters

LLM decoding is usually memory-bound: the bottleneck is streaming model weights from DRAM to SRAM, not raw compute. Speculative decoding is the main technique for absorbing that cost across multiple tokens at once, and DSpark is Liquid AI's own implementation for its LFM2.5 family, combining a parallel draft backbone, a Markov-chain sequential head, and a confidence-scheduled verifier that prunes low-confidence continuations. Shipping it with day-one support in llama.cpp and SGLang, and open-sourcing the LFM-compatible integration upstream, means the speedup is available to anyone running LFM2.5 through those two widely used inference stacks rather than staying locked to a proprietary serving layer.

Who it affects

Developers and teams already running Liquid AI's LFM2.5 models, on either a data-center GPU like the H100 or an edge device such as an Apple Silicon MacBook, stand to gain the most, particularly agentic or function-calling workloads on LFM2.5-2.6B where latency drops by 57% on average. Anyone deploying the MoE variant LFM2.5-8B-A1B on-device should expect a smaller, roughly 18% gain because of current llama.cpp Metal backend limitations with mixture-of-experts models.

How to use it

DSpark draft-model checkpoints for LFM2.5-2.6B, LFM2.5-1.2B-Instruct and LFM2.5-8B-A1B are on Hugging Face in Safetensors and GGUF format. Running them with SGLang needs a build with DSpark support for LFM2 targets (PR #31041); the target model is launched with the draft attached via speculative-decoding flags and queried at the standard OpenAI-compatible endpoint. Running them with llama.cpp needs the corresponding build (PR #27383) plus a set of spec-type flags pointing to the draft GGUF file. In both cases the block size is read automatically from the draft model's own configuration, and no pricing or licensing terms are stated in the source.

How solid is it

The benchmarks come from Liquid AI's own blog post, not an independent evaluation: throughput was measured on an M4 Max MacBook Pro with llama.cpp and Metal, and on a single H100 80GB with SGLang in BF16, across five unnamed benchmark datasets, all at batch size 1 and temperature 0. The accuracy-parity claim rests on how speculative decoding works mechanically: under greedy decoding, an accepted draft token must match the target model's own distribution, and a rejected one is replaced by the target's token, so the emitted sequence is identical to baseline greedy decoding by construction. Liquid AI names EAGLE-3 and DFlash as prior speculative-decoding approaches but does not publish a numeric comparison against either.

Risks and caveats

The reported speedups vary widely by model and workload: 52% variance for LFM2.5-1.2B-Instruct depending on the text distribution, and only an 18% average on-device gain for the MoE model LFM2.5-8B-A1B, which Liquid AI attributes to llama.cpp's current Metal MoE implementation activating more experts, and thus more weight traffic, when verifying multiple tokens per step. All figures are Liquid AI's own, gathered under fixed test conditions (batch size 1, temperature 0, a DSpark block size of 9) that may not match every production deployment, and no cost or pricing information is given.