Glonce

DSpark speeds up Liquid AI's LFM2.5 inference up to 3.2x

modelsThe Glonce editorsHugging Face🔥65Glonce rating65Impact60Novelty55Relevance92Surprise45Players45Credibility87
DSpark speeds up Liquid AI's LFM2.5 inference up to 3.2x

Liquid AI has adopted DSpark, an existing speculative-decoding technique for its LFM2.5 model family, and released matching draft-model checkpoints and day-one integration into the llama.cpp and SGLang inference engines. The company says DSpark speeds up inference by up to 3.2x on a single H100 GPU, measured with SGLang in BF16, and by up to 2.87x on-device, measured with llama.cpp and Metal on an Apple M4 Max MacBook Pro using FP16 GGUF weights and up to 256 output tokens. Both benchmarks used a DSpark block size of 9, a batch size of 1 and a temperature of 0, evaluated across five benchmark datasets.

Speculative decoding exists because the decode phase of LLM inference is usually limited by how fast weights can be streamed from memory rather than by compute: most latency comes from moving weights from DRAM into SRAM, not from doing the arithmetic. The standard fix is to let a small, fast draft model propose several candidate tokens, then have the full target model verify all of them in a single forward pass, spreading the cost of loading weights across every token that gets checked. Building on earlier approaches such as EAGLE-3 and DFlash, DSpark combines three pieces: a DFlash-style parallel backbone that produces hidden states for every draft token in one forward pass, conditioned on the target model's own context features; a lightweight sequential head, modeled as a Markov chain between neighboring tokens, that adds dependency between tokens and raises the acceptance rate later in a sequence; and a confidence-scheduled verifier that estimates each token's chance of survival and prunes the low-confidence tail of a draft sequence once checking it further would cost more than it saves.

Liquid AI trained the draft models on a larger and more varied data mix spanning supervised fine-tuning, chat, code and function-calling data. The first-generation draft models are simplified, attention-only, five layers deep, with a block size of nine; each was trained for 15 epochs over the full dataset, and the checkpoint kept was the one with the highest token-acceptance rate rather than the lowest training loss. The resulting draft models are small, around 300 million parameters each. Because a draft token is accepted only when it exactly matches the target model's own output distribution under greedy decoding, with the target model's own token substituted whenever a draft token is rejected, the final output sequence is identical to what the baseline model alone would produce; Liquid AI reports that benchmark accuracy, such as pass@1 or exact match, is therefore unchanged.

The speedup varies by model. For LFM2.5-2.6B, the gain is described as especially strong on the MacBook, and DSpark cuts latency in multi-tool, function-calling scenarios by 57% on average; Liquid AI frames this as pushing the model's on-device interactivity past what most proprietary cloud models deliver, which it puts at around 140 tokens per second depending on the dataset. For LFM2.5-1.2B-Instruct, acceptance rates vary more by dataset, and the resulting speedup swings by as much as 52% depending on the text distribution being processed. For LFM2.5-8B-A1B, a mixture-of-experts model, the acceptance rate is higher than for the two dense models, yet the on-device speedup is only about 18% on average; Liquid AI attributes the smaller gain to the current mixture-of-experts implementation in llama.cpp's Metal backend, and to the fact that verifying several draft tokens at once activates more experts, and so moves more weight traffic, than a single ordinary decode step would.

Draft-model checkpoints for all three LFM2.5 sizes, LFM2.5-2.6B-DSpark, LFM2.5-1.2B-Instruct-DSpark and LFM2.5-8B-A1B-DSpark, are published on Hugging Face in both Safetensors and GGUF format. Running them under SGLang requires a build of SGLang that carries DSpark support for LFM2 targets, tracked as PR #31041; the target model is launched with flags naming the draft model, selecting the DSpark algorithm and naming a dedicated draft-attention backend. Running them under llama.cpp requires the matching llama.cpp build, tracked as PR #27383, launched with flags that select the draft-DSpark spec type and name the draft model file; in this path, speculative decoding is described as exact, so the greedy output equals what the target model alone would produce, and per-response timings separately report how many tokens were drafted and how many were accepted.

Key facts

  • DSpark, an existing speculative-decoding technique for the LFM2.5 model family, speeds up inference by up to 3.2x on a single H100 GPU (SGLang, BF16) and up to 2.87x on-device (llama.cpp, Apple M4 Max MacBook Pro).
  • For LFM2.5-2.6B specifically, DSpark cuts function-calling and multi-tool latency by 57% on average.
  • DSpark combines a DFlash-style parallel backbone, a Markov-chain sequential head, and a confidence-scheduled verifier that prunes low-confidence draft tokens.
  • Because rejected draft tokens are replaced by the target model's own token under greedy decoding, DSpark's output is identical to the baseline model's, so benchmark accuracy such as pass@1 is unchanged.
  • Draft-model checkpoints for three LFM2.5 sizes are published on Hugging Face in Safetensors and GGUF, with day-one integration into llama.cpp and SGLang.

Why it matters

The decode phase of LLM inference is usually limited by how fast weights move through memory rather than by compute, so speculative decoding, letting a small draft model propose tokens that a larger target model verifies in one pass, has become one of the standard ways to cut inference cost without touching the target model's own weights. DSpark is an existing version of that idea for LFM2.5, built by combining three techniques into one drafter: a parallel backbone in the style of DFlash that produces every draft token's hidden state in a single pass, a small sequential head that models neighboring tokens as a Markov chain to keep later tokens more likely to be accepted, and a confidence-scheduled verifier that gives up early on a draft sequence once continuing to check it would cost more than it would save. Because rejected tokens are simply swapped for the target model's own output under greedy decoding, the approach is presented as exact rather than approximate: it changes speed, not the answer.

Who it affects

The release covers three sizes of LFM2.5: the 2.6B and 1.2B-Instruct dense models and the 8B-A1B mixture-of-experts model, each with its own DSpark draft model. Anyone running these models in the cloud, benchmarked here on a single 80GB H100 GPU through SGLang, or locally, benchmarked on an Apple M4 Max MacBook Pro through llama.cpp with Metal, gets a drop-in speedup without retraining or requantizing the target model itself. The gains are uneven across the three sizes: LFM2.5-2.6B benefits the most, with the 57% average cut in multi-tool latency; LFM2.5-1.2B-Instruct's speedup swings by as much as 52% depending on the text being processed; and LFM2.5-8B-A1B, despite a higher token-acceptance rate than the two dense models, sees only an 18% average on-device improvement, which Liquid AI attributes to how llama.cpp's Metal backend currently handles mixture-of-experts models and to the extra expert traffic that verifying several draft tokens at once creates.

How to use it

Draft-model checkpoints, LFM2.5-2.6B-DSpark, LFM2.5-1.2B-Instruct-DSpark and LFM2.5-8B-A1B-DSpark, are published on Hugging Face in Safetensors and GGUF formats. Using them with SGLang needs a build that carries DSpark support for LFM2 targets, tracked as PR #31041: the target model is launched with flags that point to the draft model, select the DSpark algorithm and name a draft-attention backend. Using them with llama.cpp needs the matching llama.cpp build, tracked as PR #27383, launched with flags that name the draft model file and select the draft-DSpark spec type; each response's timing output separately reports how many tokens were drafted and how many were accepted. Both integrations are described as open-sourced upstream, and no pricing or cost information is given for the checkpoints or the code.

How solid is it

Every figure here comes from Liquid AI's own blog post and its own measurements across five benchmark datasets; the post names no independent or third-party benchmark that reproduces the speedups, and it gives no exact acceptance-rate percentages for any of the three models, only relative statements about how much they vary. What is more rigorously stated is the accuracy claim: because a draft token is accepted only when it matches the target model's own distribution exactly, with the target model's token substituted on any rejection, the emitted sequence under greedy decoding is, by construction, identical to what the unaccelerated model would produce, so pass@1 or exact-match scores do not move. That is a structural guarantee about correctness, not a benchmark result, and it is separate from the throughput multipliers, which are self-reported and vary noticeably by model and by dataset.

Risks and caveats

None of the throughput numbers come with an absolute baseline: the post states multipliers and percentage changes (3.2x, 2.87x, 57%, 52%, 18%) but never the underlying tokens-per-second or millisecond figures they were computed from for LFM2.5 itself; the one absolute figure given, around 140 tokens per second, describes other, proprietary cloud models used only as a comparison point, not LFM2.5's own baseline. The speedup is also uneven and workload-dependent: LFM2.5-1.2B-Instruct's gain swings by as much as 52% across datasets, and the mixture-of-experts model LFM2.5-8B-A1B sees only an 18% average improvement on-device despite a higher acceptance rate, a gap Liquid AI attributes to llama.cpp's current Metal backend rather than to DSpark itself. Using any of this requires building SGLang or llama.cpp from specific pull requests rather than a released stable version, and no pricing information is given beyond the checkpoints and integration code being open-sourced.