AMPLE-Math benchmark shows reference solutions add little to self-distillation

On-policy self-distillation (OPSD) trains a language model using a frozen copy of itself as the teacher, and that teacher can be handed extra material to work with, such as a final answer or a full worked solution. It has seemed intuitive that this privileged information gives the student more to learn from, but nothing had isolated how much of the resulting improvement it actually contributes on top of ordinary distillation.

To test this, the authors built AMPLE-Math, a reusable benchmark of 5,319 math problems, each rewritten into six reasoning views that all share the same final answer. That design lets a reference-bearing view be compared directly against matched reference-free distillation on the identical problem.

Using a thinking-enabled teacher to supervise direct-response student rollouts, the authors found that reference-free distillation alone already accounts for much of Qwen3-1.7B's improvement under thinking-enabled evaluation, both on the AMPLE-Math problems themselves and on external benchmarks. Adding a privileged reference produced only modest extra gains in Qwen, strongest when the reference was a polished, finished solution rather than a raw trace. For SmolLM3-3B, giving the teacher complete reasoning traces added about two percentage points over reference-free distillation, measured at training step 50.

These benefits are not universal: they depend on which student model is being trained. At the same training checkpoint, swapping short direct-response rollouts for long thinking-enabled ones turned the gains into losses in both Qwen3-1.7B and SmolLM3-3B, even though the problems, the references and the evaluation setup were held fixed. Separate experiments with teacher profiles and matched loss interventions in Qwen showed that changing which tokens receive supervision can leave student behavior largely unchanged.

The authors conclude that OPSD mainly improves a student's access to reasoning capability it already has, through parameters shared between its direct-response and thinking-enabled modes. On that reading, what matters about a privileged reference is not how much of the solution it reveals, but what it adds to this transfer between modes.

Key facts

AMPLE-Math is a new reusable suite of 5,319 math problems with six reasoning views sharing the same answer, built to isolate what privileged information adds to on-policy self-distillation.
Reference-free distillation already accounts for much of Qwen3-1.7B's improvement under thinking-enabled evaluation, both in domain and on external benchmarks.
Adding a privileged reference gave only modest extra benefit in Qwen, strongest with a polished solution; complete traces added about two percentage points for SmolLM3-3B at training step 50.
Swapping short direct-response rollouts for long thinking-enabled ones turned gains into losses in both model families at the same checkpoint.
Changing token-level supervision through teacher profiles and matched loss interventions in Qwen left student behavior largely unchanged.

Why it matters

On-policy self-distillation is a training method for squeezing more capability out of a model using only a frozen copy of itself as teacher, and it is tempting to assume that giving that teacher privileged information, such as a reference answer or a worked solution, automatically helps more. This paper is a controlled test of that assumption: AMPLE-Math holds the problem and the final answer fixed while varying only how much of the reasoning is revealed to the teacher. It matters because it separates what actually drives the improvement: much of it comes from distillation itself, not from the privileged reference.

Who it affects

Teams building post-training or self-improvement pipelines for language models, particularly anyone using on-policy self-distillation with reference solutions or answer keys as a design choice. The paper's own experiments cover two specific model families, Qwen3-1.7B and SmolLM3-3B, so results speak most directly to teams working with models of similar scale. It affects decisions about whether it is worth the engineering effort to assemble privileged references in the first place.

How to use it

AMPLE-Math is described as reusable, so a team can run its own self-distillation setup against the same 5,319-problem, six-view suite before deciding whether to invest in gathering privileged references. The practical takeaway: check a reference-free distillation baseline first, since it may already capture much of the available gain, and only add a polished reference solution if a checkpoint-level test shows a measurable delta. That delta is student-dependent: modest for Qwen3-1.7B, about two percentage points for SmolLM3-3B at step 50.

How solid is it

The study uses matched comparisons within one benchmark, holding the problem and the final answer fixed across six reasoning views, which is a stronger design than comparing unrelated setups. It reports results across two model families, checks both in-domain and external benchmarks, and adds a separate ablation on teacher profiles and loss interventions. Against that, no absolute accuracy figures appear in the available text, only deltas, so the underlying size of the effect relative to baseline performance cannot be independently judged, and no author names or institutions are given.

Risks and caveats

The benefit of privileged information is reported as inconsistent: it depends on which student model is trained, is modest for Qwen3-1.7B, and reaches roughly two percentage points only for SmolLM3-3B under one specific measurement, complete traces at step 50. Swapping rollout length reversed the direction of the effect entirely in both families, which suggests the results are sensitive to configuration choices beyond simply the presence of a reference. Generalization beyond these two model families and this math-problem domain is not established in the available text.

“The value of a privileged reference is what it adds to this cross-mode transfer, not how much of the solution it reveals.”

— the authors