Glonce

Multiverse Computing's 4-bit model beats its own full-precision version

researchThe Glonce editorsHugging Face🔥74Glonce rating74Impact65Novelty75Relevance100Surprise55Players45Credibility85
Multiverse Computing's 4-bit model beats its own full-precision version

Multiverse Computing published a paper and blog post introducing Quantization-Aware Healing (QAH), a technique for recovering a large language model after it has gone through both structural compression (removing layers, heads or neurons) and quantization down to 4 bits. Applied to a GPT-OSS 120B model that was first structurally compressed to 60B parameters and recovered in bfloat16 (full 16-bit precision), then re-quantized to MXFP4 under QAH, the resulting 4-bit model beats that same 60B model's own bfloat16 checkpoint on 7 of 9 benchmarks tested. It loses on the other two, MMLU-Pro and SciCode, by less than a point and a half. Its largest gains land on the capabilities compression usually damages most: it gains 7.4 points on the AA-LCR long-context reasoning benchmark and 5.6 points on the AIME 2025 math benchmark.

The QAH model also holds up against the original, uncompressed 120B teacher model. Despite running at half the teacher's parameter count and roughly a quarter of its weight memory, it beats the full-size teacher on LiveCodeBench (66.5 versus 66.0) and comes within 1.6 points of it on GPQA Diamond (67.4 versus 69.0). The largest remaining gap to the teacher is on AA-LCR, the extreme long-context benchmark, where the authors say the capacity lost to compression is hardest to recover.

Two established recipes already exist for healing a quantized model. Quantization-aware training (QAT) fine-tunes the compressed model on a task loss through a noisier, low-precision forward pass; the authors say this is costly, since it re-runs an already expensive multi-stage post-training process, and can become unstable if training continues too long past its best point. Quantization-aware distillation (QAD) instead distills a frozen full-precision teacher into the quantized student through a KL-divergence loss on output logits; this works well when quantization is the only change, because a genuine full-precision version of the exact same model exists to act as teacher. But once a model has also been structurally compressed, no independently trained full-precision version of the smaller architecture exists, only the recovered bfloat16 checkpoint itself, which is already a degraded approximation, and distilling from it caps the student's accuracy at that checkpoint's own ceiling. QAH removes that ceiling by distilling directly from the original, pre-compression, full-size teacher instead of the recovered one. Teacher and student do not even share an architecture, but because a teacher's output distribution is architecture-agnostic, the size and shape mismatch does not block the transfer.

To isolate the effect of the loss function itself, Multiverse Computing also ran a direct head-to-head between QAH and QAT under matched conditions, quantizing a separate GPT-OSS 9B model to MXFP4 and tracking average accuracy across MMLU-Pro, LiveCodeBench and GPQA Diamond over the course of training. Both methods reach a similar peak, 54.9 for QAH against 54.6 for QAT. But QAH reaches that peak in about 100 training steps, roughly 7 times faster than QAT's 700 steps, and then stays within about two points of its peak for the rest of training. QAT collapses once past its peak, shedding nearly 19 points by step 1,200. The authors attribute this to the loss itself: KL distillation against a fixed teacher gives the student no incentive to drift once it matches the teacher, while a cross-entropy task loss keeps pushing the student toward hard labels and can erode capability over time. The practical consequence, per the authors, is that a QAT checkpoint needs careful early stopping against a held-out signal to avoid shipping a model that has already started to degrade, whereas a sufficiently trained QAH checkpoint can be served safely because it does not drift.

On efficiency, the 4-bit QAH model uses roughly 4 times less weight memory than the bfloat16 student it was distilled from, and at half the 120B teacher's parameter count it roughly halves compute per token; for model families that ship natively in bfloat16 rather than 4-bit, the combined parameter and precision reduction would be closer to 8 times less compute per token. To make the healing process work at long context, the training corpus includes documents up to 32,000 tokens, handled with a memory-efficient chunked KL-divergence loss carried over from a companion paper on efficient distillation, which computes the divergence one sequence slice at a time instead of materializing the full vocabulary-by-sequence grid. The work is part of Multiverse Computing's broader research into making large models smaller and cheaper to run without giving up the capabilities that make them useful.

Key facts

  • Multiverse Computing's Quantization-Aware Healing (QAH), applied to a GPT-OSS 120B model compressed to 60B parameters and quantized to MXFP4, beats its own bfloat16 checkpoint on 7 of 9 benchmarks, including gains of +7.4 on AA-LCR and +5.6 on AIME 2025.
  • Despite running at half the parameter count and roughly a quarter of the weight memory of the original 120B teacher model, the QAH model beats that teacher on LiveCodeBench (66.5 vs. 66.0) and trails it by only 1.6 points on GPQA Diamond (67.4 vs. 69.0).
  • In a matched head-to-head on a GPT-OSS 9B model, QAH matched quantization-aware training's peak accuracy (54.9 vs. 54.6) in about 100 steps, roughly 7 times faster than QAT's 700 steps, and did not degrade afterward, while QAT lost nearly 19 points by step 1,200.
  • QAH works by distilling directly from the original, pre-compression full-precision model rather than from the already-degraded recovered checkpoint that other healing methods use as teacher.
  • The 4-bit QAH model uses roughly 4 times less weight memory than its bfloat16 counterpart, and model families that ship natively in bfloat16 could see close to 8 times less compute per token if compressed and healed this way.

Why it matters

QAH breaks the usual assumption that shrinking a model to 4 bits costs accuracy. It shows that a model which has been both structurally compressed and quantized can end up smaller, cheaper to run, and more accurate than the full-precision version it was derived from, inverting the normal relationship between a 4-bit model and its 16-bit source. Most efficiency pipelines treat the final healing step as damage control after compression and quantization; QAH instead treats quantization as a second full pass of distillation against the original model, an opportunity to teach the model rather than a tax paid for efficiency.

Who it affects

Teams and organizations that need to run large language models more cheaply, and engineers building compression and quantization pipelines who currently have to choose between quantization-aware training and quantization-aware distillation for recovering a model. It is most directly relevant to open-weight models such as GPT-OSS, where structural compression, cutting layers, heads or neurons, is combined with low-bit quantization such as MXFP4.

How to use it

Multiverse Computing describes QAH in a paper and an accompanying blog post, covering the healing pipeline, the chunked KL-divergence implementation used for long-context healing, and distributed-training findings. The post does not state whether the QAH model, its weights, or its code will be released. Readers wanting the technical details are pointed to the full paper, or invited to contact Multiverse Computing's team about applying compression and healing to their own models; no price or licensing terms are given.

How solid is it

The claim is backed by benchmark numbers rather than assertion alone: 9 benchmarks for the 120B-to-60B result, plus a separate controlled head-to-head against QAT on a 9B model that isolates the effect of the loss function specifically. The gains are broad (7 of 9 benchmarks won, plus advantages in training speed and stability) and the losses where they occur are small, under a point and a half. That said, this is Multiverse Computing's own paper and blog post about its own method, using its own choice of benchmarks; no independent replication or third-party evaluation is reported, and no individual author names are given, only the collective 'we' of the company.

Risks and caveats

No cost or GPU-hour figures are given, only step counts and benchmark-point differences, so the training cost of QAH relative to QAT beyond step counts is unclear. QAH still trails the bfloat16 checkpoint on two benchmarks, MMLU-Pro and SciCode, and trails the full-size 120B teacher on the AA-LCR long-context benchmark, the gap the authors say is hardest to close. The results so far come from one model family, GPT-OSS, and one company's internal benchmarking, with no stated release of code or weights for independent verification.

“This inverts the usual relationship between a 4-bit model and the 16-bit model it came from.”

— Multiverse Computing