Qwen3 study: 6-bit weights plus up to four option orders is the best budget split

A paper listed on Hugging Face asks two linked questions about small local language models. Storing weights in fewer bits saves memory, and averaging a few reorderings of the answer options reduces sensitivity to option order. Does the first make the second worse, and which is the cheaper way to spend a fixed budget?
The setup is small. The authors use Qwen3-0.6B and Qwen3-1.7B with simulated round-to-nearest weight quantization at 8, 6, 4 and 3 bits, with fp16 as the reference. Answers are read from the logits of the letters A to D on four benchmarks: ARC-Easy, ARC-Challenge, MMLU and HellaSwag, 300 items each.
On accuracy, 6-bit weights lose nothing against fp16. They score +1.8 points (range +1.0 to +2.5) higher on the 0.6B model and +0.9 points (+0.3 to +1.5) higher on the 1.7B model, a gain the authors say they do not explain. 8-bit weights change accuracy by -0.7 (-1.1 to -0.3) and -0.4 (-0.7 to -0.2) points for the two models.
On order sensitivity, 4-bit weights raise the answer-slot skew from 0.15 to 0.53 on the 0.6B model and from 0.07 to 0.12 on the 1.7B model. As a control, the authors add Gaussian noise at the same weight signal-to-noise ratio. That gives skews of 0.15 and 0.37 (two draws) and 0.19 (one draw). So the control is inconclusive on whether rounding adds slot skew beyond equal-power noise. Flip rates rise with quantization but stay below an accuracy-matched baseline of independent uniform errors.
Averaging four cyclic orders recovers +12.6 points at 4 bits on the 0.6B model, against +5.3 points at fp16.
To compare the two levers, the authors price bits and passes in one machine-independent unit: weight bytes read times passes. In cost-matched pairs, extra option orders beat extra bits only when the weights have 6 or more bits. The decision table reduces to one rule: use 6-bit weights and spend the rest of the budget on up to four orders, one pass each.
The authors state the limits themselves: everything is simulated on two small models with the simplest quantizer. They also say a title-level search found no earlier study of quantization together with option order.
Key facts
- On Qwen3-0.6B and Qwen3-1.7B, 6-bit weights score +1.8 and +0.9 points above fp16, a gain the authors do not explain; 8-bit weights change accuracy by -0.7 and -0.4 points.
- At 4 bits, answer-slot skew rises from 0.15 to 0.53 (0.6B) and from 0.07 to 0.12 (1.7B), but a Gaussian-noise control is inconclusive on whether rounding itself is the cause.
- Averaging four cyclic option orders recovers +12.6 points at 4 bits on the 0.6B model, versus +5.3 points at fp16.
- Cost is counted as weight bytes read times passes; extra orders beat extra bits only at 6 or more bits.
- The resulting rule: use 6-bit weights and spend the rest of the budget on up to four orders, one pass each.
Why it matters
Small local models are often run quantized to save memory, and multiple-choice results are known to shift with the order of the options. This paper puts the two together and puts them on one cost scale, so the choice between more bits and more passes becomes a concrete rule rather than a guess. The authors say a title-level search found no earlier study of quantization together with option order.
Who it affects
People who run small models such as Qwen3-0.6B and Qwen3-1.7B locally and score them on multiple-choice benchmarks of the ARC, MMLU and HellaSwag kind, where answers are read from the letters A to D. The findings are about those two models only.
How to use it
The paper's own decision rule is to use 6-bit weights and spend the remaining budget on up to four option orders, one pass each. Cost is counted as weight bytes read times passes. The source reports that extra orders beat extra bits only when weights have 6 or more bits; below that, the comparison does not favour extra orders.
How solid is it
The numbers carry ranges: for example the 6-bit gain of +1.8 (+1.0 to +2.5) on the 0.6B model and +0.9 (+0.3 to +1.5) on the 1.7B model. The evaluation uses 300 items per benchmark across four benchmarks. The authors flag one result as inconclusive: the Gaussian-noise control cannot say whether rounding adds slot skew beyond equal-power noise. They also do not explain why 6-bit weights beat fp16. The text read here is the paper's abstract.
Risks and caveats
Everything is simulated, on two small models, with the simplest quantizer (round-to-nearest). No real quantized kernels, hardware measurements or latency figures are reported, since cost is a machine-independent unit. Only Qwen3-0.6B and Qwen3-1.7B are covered, and no larger models or other quantizers are mentioned. The rule may not carry over to other models or real deployments. Treat the 6-bit recommendation as a result for this setup.
“use 6-bit weights and spend the rest of the budget on up to four orders, one pass each”
— Paper abstract, Hugging Face papers page