Glonce

Unsloth releases Dynamic v3.0 GGUFs, over 10% more accurate than other providers

toolsThe Glonce editorsHacker News🔥75Glonce rating75Impact65Novelty65Relevance95Surprise40Players55Credibility85
Unsloth releases Dynamic v3.0 GGUFs, over 10% more accurate than other providers

Unsloth has released Dynamic v3.0, the next iteration of its GGUF quantization method and an update to an early preview version it had already shared. The first quants built with the new method are for Qwen3.8-27B, and Unsloth says they deliver more than 10% better top-1% accuracy than every other provider's quants at the same size. The new GGUFs work with most inference engines, including llama.cpp and Unsloth Desktop. Unsloth says Dynamic v3.0 preserves more model quality at a given size overall, with stronger results on two internal metrics it uses, Divergence-300 @32 and KL Divergence, and reports that the community has downloaded its Unsloth Qwen3.8 quants over 5.1 million times in five days.

The new methodology is built on a higher-quality imatrix calibration dataset drawn from more diverse sources and refined for agentic coding, chat and multilingual performance, along with improved layer selection and additional quantization techniques meant to preserve model quality. Unsloth says it does not train on the calibration dataset itself and does not use quantization-aware training or distillation, known as QAT or QAD; everything is done through post-training quantization, or PTQ. The imatrix file used to build the quants is published for the community to test, evaluate and reuse, and Unsloth is encouraging researchers and developers to build their own variants and fine-tunes of Qwen3.8 from the released quants and imatrix file.

On size, Unsloth removed the MTP module from smaller quants under the UD-Q2_K_XL size, 8.37GB and lower, saving around 500MB of disk space; a separate Q4_0 MTP module stays available for anyone who needs it. A new, smaller 1-bit variant, UD-IQ1_S, comes in at 6.2GB without the MTP module and retains around 72% top-1% accuracy while being 89% smaller. The UD-Q2_K_XL quant, at 9.83GB, is around 8% more accurate on top-1% than the next-best competing quant, and Unsloth says it produced a working HTML program with only one small JavaScript bug, where an earlier, unspecified baseline would previously break entirely.

Unsloth argues that the usual top-1% accuracy metric, an argmax over a single predicted token, does not reflect real inference well, so it built a new evaluation called Divergence-300 @32: 300 held-out examples, excluded from the calibration data and drawn from Terminal-Bench 2.1, DeepSWE, Harbor, MathArena 2025-26 and non-Latin or long-document prompts, over which it runs greedy argmax decoding for 32 tokens and compares BF16 against every quant and provider. Separately, Unsloth ran KL Divergence benchmarks across providers and reports that its UD-3 quants get up to 10% extra top-1% accuracy at the same disk space, an advantage most visible on smaller quant sizes; the disk-space comparisons exclude the MTP head for every provider, to keep the comparison fair. Against the older UD-2 method, on unseen Wikitext and code data, KL Divergence improved substantially for smaller quants but much less for larger ones, so Unsloth is keeping UD-2 for its larger quants for now while it works on closing that gap. On overfitting, Unsloth says it uses separate datasets for calibration and testing, that its purely post-training approach makes overfitting less of a concern than QAT or QAD approaches, and that the Divergence-300 @32 results on unseen prompts show the new UD-3 methods are not overfitting to the calibration data.

Key facts

  • Unsloth released Dynamic v3.0, and its first quants, for Qwen3.8-27B, deliver more than 10% better top-1% accuracy than every other provider's quants at the same size.
  • Downloads of Unsloth's Qwen3.8 quants have passed 5.1 million in five days.
  • Removing the MTP module from quants under the UD-Q2_K_XL size (8.37GB and lower) saves around 500MB of disk space; a separate Q4_0 MTP module stays available for anyone who needs it.
  • The new 1-bit UD-IQ1_S quant is 6.2GB without the MTP module and retains around 72% top-1% accuracy while being 89% smaller.
  • Unsloth built a new benchmark, Divergence-300 @32, testing 32 tokens of greedy decoding against BF16 on 300 held-out prompts, arguing that ordinary top-1% accuracy is a poor proxy for real inference quality.

Why it matters

Quantization is what lets people run large language models on ordinary hardware, at some cost in accuracy. Dynamic v3.0 pushes that tradeoff further: Unsloth says its Qwen3.8-27B quants are more than 10% better on top-1% accuracy than every other provider's quants at the same size, with the biggest gains concentrated at the smaller quant sizes. The new UD-IQ1_S 1-bit quant illustrates the range: at 6.2GB and 89% smaller, it still keeps around 72% top-1% accuracy, so a large model can run on far less memory and disk than before. Unsloth's own download figures, over 5.1 million for its Qwen3.8 quants in five days, point to how much demand already exists for running capable open models through tools like llama.cpp and Unsloth Desktop.

Who it affects

Anyone running open-weight models locally through llama.cpp or Unsloth Desktop can pull the new Qwen3.8-27B Dynamic v3.0 quants directly; the smallest ones, like the 6.2GB UD-IQ1_S, bring a 27-billion-parameter model down to a size that fits on far more modest hardware. Unsloth is also publishing the imatrix calibration file it used to build the quants, so researchers and developers can test, evaluate or build their own variations and fine-tunes of Qwen3.8 on top of it rather than starting from scratch. The scale of the 5.1 million downloads Unsloth reports for its Qwen3.8 quants in five days is itself evidence of an active audience: developers already running or experimenting with the model locally.

How to use it

The Qwen3.8-27B Dynamic v3.0 GGUFs are out now and work with most inference engines, including llama.cpp and Unsloth Desktop. They span a range of sizes: the 1-bit UD-IQ1_S at 6.2GB without the MTP module, for the tightest hardware budgets, up to the UD-Q2_K_XL at 9.83GB, which Unsloth says is around 8% more accurate on top-1% than the next-best competing quant. Unsloth removed the MTP module from quants under the UD-Q2_K_XL size (8.37GB and lower) to save around 500MB of disk space, while keeping a separate Q4_0 MTP module available for anyone who wants it added back. Unsloth has also published the imatrix calibration file behind the release, so anyone can inspect it, test it, or use it to build their own quant variants or fine-tunes of Qwen3.8.

How solid is it

This is Unsloth's own account of its own benchmarks, published on its own site with no named author, so the accuracy comparisons against "every other provider" are self-reported rather than independently verified. Unsloth does describe its methodology in some detail: it argues that top-1% accuracy, an argmax over a single predicted token, is a weak signal for real inference quality, so it built a new evaluation, Divergence-300 @32, from 300 examples held out of the calibration data (drawn from Terminal-Bench 2.1, DeepSWE, Harbor, MathArena 2025-26, and non-Latin or long-document prompts) and compares 32 tokens of greedy decoding against the BF16 model across every quant and provider. It also ran KL Divergence benchmarks separately and reports that UD-3 quants get up to 10% extra top-1% accuracy at the same disk space, again concentrated at smaller sizes; on unseen Wikitext and code data, the improvement over the older UD-2 method was large for small quants but much smaller for large ones, so Unsloth says it is still shipping UD-2 for its largest quants while it works on closing that gap. To guard against overfitting, Unsloth says it keeps calibration and testing data separate and uses only post-training quantization, with no quantization-aware training or distillation, and that the unseen-prompt Divergence-300 @32 results show the new UD-3 methods hold up. As one concrete example, Unsloth says the 9.83GB UD-Q2_K_XL quant produced a working HTML program with only one small JavaScript bug, where it says an earlier, unspecified baseline would previously fail entirely, though that is a single anecdote rather than a benchmark result.

Risks and caveats

The headline claim, more than 10% better top-1% accuracy for Qwen3.8-27B Dynamic v3.0 quants against every other provider at the same size, and the separate claim of up to 10% extra top-1% accuracy for UD-3 quants generally from the KL Divergence benchmarks, come from different parts of the post and different comparisons; they should not be read as the same number restated twice. The gains are also not uniform across the lineup: Unsloth's own numbers show much smaller KLD improvement over UD-2 on larger quants than on smaller ones, so Unsloth is still using the older UD-2 method for its largest quants rather than UD-3. Some of the smaller figures in the post lack a stated baseline: the "89% smaller" UD-IQ1_S figure and the "62% smaller" figure cited for a separate Kimi-K3 example do not say in the same sentence what they are smaller than, and the "next best" quant that UD-Q2_K_XL is compared against for its 8% accuracy edge is not identified. No calendar date is given for the release beyond "today," and the five-day download window is not tied to specific dates either.

“Top-1% is an argmax on 1 prediction, so it's not really effective on gauging actual inference.”

— Unsloth, in the Dynamic v3.0 announcement