Glonce

Reflective Recovery teaches LLMs to self-correct reasoning errors

researchThe Glonce editorsarXiv🔥74Glonce rating74Impact68Novelty76Relevance98Surprise58Players22Credibility78

Researchers propose Reflective Recovery, a self-supervised fine-tuning method aimed at a specific failure mode in reasoning models. Standard imitation learning trains LLMs only on perfect reasoning trajectories, and the authors argue this approach suffers from what they call Scaling Collapse: when the pool of problems is limited, adding more correct examples stops producing further gains. A related problem shows up at inference time: a model cannot guarantee every intermediate reasoning step is correct, and once an error appears, the model often fails to recover and gets further misled by the mistakes it already made.

Reflective Recovery addresses this by training directly on failure. The method takes the initial segments of reasoning trajectories that ended in a wrong answer, concatenates them with the original prompt, and uses that combination to guide the model toward a valid solution. Because those initial segments are likely to already contain an error, training on them teaches the model to recognize a mistake mid-reasoning and correct course, without any external critic model or reward model checking its work.

On DeepSeek-R1-Distill-Qwen-7B, the method raises accuracy from 30.0% to 37.5% on AIME 2025 and from 37.6% to 47.8% on Minerva. Beyond the benchmark gains, the authors report that Reflective Recovery breaks the scaling collapse barrier they identified and produces emergent self-correction behavior in the model, which they frame as a shift from training that rewards memorizing correct outcomes to training that rewards the reasoning process itself.

Key facts

  • Reflective Recovery is a self-supervised fine-tuning method that converts failed reasoning trajectories into recovery training data instead of discarding them.
  • It uses only the initial (likely erroneous) segment of a failed trajectory, paired with the original prompt, to teach the model to recognize and correct mistakes mid-reasoning.
  • The method needs no external critic model or reward model to identify or grade the errors.
  • On DeepSeek-R1-Distill-Qwen-7B, accuracy rises from 30.0% to 37.5% on AIME 2025 and from 37.6% to 47.8% on Minerva.
  • The authors report the method breaks the Scaling Collapse they identify in imitation learning on perfect-trajectory data, and produces emergent self-correction behavior.

Why it matters

Fine-tuning LLMs on curated, perfect reasoning traces is the standard recipe for improving reasoning, but the authors identify a ceiling on it: with a limited problem set, adding more correct examples stops helping, a pattern they name Scaling Collapse. Reflective Recovery targets that ceiling directly by mining a resource that imitation learning throws away: the model's own failed attempts. Training on the point where a trajectory goes wrong, rather than only on trajectories that never go wrong, is a different training signal, and the paper reports it breaks through the collapse rather than just nudging accuracy up.

Who it affects

The direct audience is researchers and engineers building reasoning-focused LLMs, particularly teams fine-tuning open models like DeepSeek-R1-Distill-Qwen-7B on math and reasoning benchmarks such as AIME and Minerva. It also matters to anyone trying to build self-correction into a model without standing up a separate critic or reward model, since that is the exact overhead the method is designed to remove.

How to use it

The recipe as described: take failed reasoning trajectories, cut them down to their initial segment, concatenate that segment with the original prompt, and use the pair to train the model toward a correct completion. No reward model or external critic is required at any point, since the training signal comes from the trajectory's own known failure rather than an external judgment of quality.

How solid is it

The result is a single reported benchmark comparison: on one model, DeepSeek-R1-Distill-Qwen-7B, accuracy rises from 30.0% to 37.5% on AIME 2025 and from 37.6% to 47.8% on Minerva, gains of 7.5 and 10.2 percentage points. The abstract states the method was evaluated on "extensive benchmarks" but does not name or report results for benchmarks beyond these two, and it gives no author names, institution, or venue, so independent replication or scrutiny of the broader claim is not yet possible from this text alone.

Risks and caveats

The headline numbers come from a single 7B-parameter model; whether the gains hold at larger scale or on other model families is not addressed in the available text. The claim of "emergent self-correction behavior" and a "paradigm shift" toward process-oriented training is the authors' own characterization of their analyses, not an independently verified property. No comparison figures are given against the external critic or reward model approaches the method is positioned as an alternative to, so the size of that advantage, if any, is not established here.