Glonce

RISE recursively distills an LLM's own training into a teacher

researchThe Glonce editorsHugging Face🔥71Glonce rating71Impact63Novelty77Relevance98Surprise45Players25Credibility82
RISE recursively distills an LLM's own training into a teacher

On-policy distillation (OPD) gives language models dense, per-token supervision during post-training, but the authors say its effectiveness is bottlenecked by teacher quality: an external teacher brings distribution mismatch, and self-distillation with privileged conditioning is capped by how much the model's in-context learning can carry. RISE, short for Recursive Improvement via Self-Extrapolating Policy Distillation, is proposed as a fix that needs neither an external model nor privileged conditioning.

RISE builds its own synthetic teacher straight from the model's reinforcement learning with verifiable rewards (RLVR) trajectory. It extrapolates the displacement between the current checkpoint and a trailing anchor, either in parameter space or in output logit space, and turns that sparse, outcome-driven parameter update into a dense, token-level training target.

The two training signals then run as a loop rather than as separate stages: outcome rewards from RLVR ground the extrapolation toward correct reasoning, while the extrapolated teacher refines the model's token-level decisions. Because the teacher is rebuilt every iteration as the student model improves, the authors describe distillation here as a recursive improvement mechanism rather than a one-shot compression step applied once against a fixed teacher.

The authors report experiments across mathematical reasoning, multi-domain STEM problems, code generation, and multi-turn agentic tasks, and say RISE outperforms both RLVR-only training and on-policy self-distillation in every one of those settings. The text made available for this story is the paper's abstract; it does not include the numeric benchmark results, model sizes or families, benchmark names, author affiliations, or any code or weights release information.

Key facts

  • RISE builds a synthetic teacher for on-policy distillation directly from the model's own RLVR training trajectory, by extrapolating the displacement between the current checkpoint and a trailing anchor in parameter space or output logit space.
  • It converts a sparse, outcome-driven RL parameter update into a dense, token-level distillation target, without any external teacher model or privileged conditioning.
  • RISE runs RLVR and on-policy distillation as a complementary loop: outcome rewards ground the extrapolation toward correct reasoning, and the extrapolated teacher refines token-level decisions.
  • The teacher is refreshed every iteration as the student model improves, so the authors frame distillation as a recursive improvement mechanism rather than a one-shot compression step.

Why it matters

Post-training methods that distill from a teacher model usually need either a stronger external model, which can mismatch the student's own output distribution, or in-context privileged conditioning, which runs into the limits of what a model can carry in its context. RISE sidesteps both by manufacturing its teacher signal from the model's own RLVR trajectory, converting a training update that would otherwise only touch the final outcome into supervision at every token.

Who it affects

The proposal targets teams doing LLM post-training with reinforcement learning with verifiable rewards, particularly ones where a suitably strong external teacher for distillation is not available or would introduce distribution mismatch. It is a training-recipe contribution rather than a deployable product or a released model.

How to use it

RISE is described as a training procedure: extrapolate the checkpoint-to-anchor displacement to synthesize a teacher, then run RLVR and on-policy distillation together in a loop, refreshing the synthetic teacher each iteration. The text made available does not state whether code, model weights, or a dataset have been released, so no availability or licensing detail can be given.

How solid is it

The evidence is the authors' own reported experiments, spanning mathematical reasoning, multi-domain STEM, code generation, and multi-turn agentic tasks, in which they say RISE outperforms both RLVR-only training and on-policy self-distillation across all of those settings. The text available for this story is the paper's abstract, which does not carry the actual benchmark numbers, model sizes, or benchmark names behind that claim.

Risks and caveats

This account rests on the abstract as the self-reported summary of the authors' own experiments; no independent numbers are available to check the outperformance claim against. The text does not name an institution or author affiliation, give model sizes or families, or state whether code, weights, or a dataset will be released, so none of that can be reported here.

“its effectiveness is bottlenecked by teacher quality: external teachers suffer from distribution mismatch, while self-distillation with privileged conditioning is limited by in-context learning capacity”

— RISE paper, arXiv 2609.05295