Glonce

Mixed SFT beats next-chunk reasoning RL with over 60x less compute

researchThe Glonce editorsHugging Face🔥74Glonce rating74Impact65Novelty70Relevance100Surprise65Players25Credibility80
Mixed SFT beats next-chunk reasoning RL with over 60x less compute

No-CoT data, such as worked solutions and textbook derivations, contains reasoning-rich content but no explicit chain-of-thought annotations. A recent line of work trains on this data with next-chunk reasoning RL: the model is trained to generate an implicit reasoning trace and is rewarded based on how well that trace helps it predict the next chunk of the original text. The catch, according to a paper submitted to arXiv on August 24, 2026 by Yinhao Tang, Youqing Fang, Yanan Sun, Jiangning Liu, Ziyi Wang, Xun Zhao, Weiming Zhang, Bin Liu, Kuikun Liu, Wenwei Zhang and Kai Chen, is that prior evaluations of next-chunk reasoning RL mostly compared it against conventional SFT baselines. That left it unclear whether the method's reported gains actually came from the RL formulation itself, or simply from exposing the model to no-CoT data more effectively.

To separate those two explanations, the authors ran a controlled study and introduced a simple alternative they call Mixed SFT: a single supervised fine-tuning stage that trains jointly on no-CoT and long-CoT data. Despite being simpler than next-chunk reasoning RL, Mixed SFT reached a clearly higher performance ceiling after the subsequent RLVR (reinforcement learning with verifiable rewards) stage, while needing over 60 times less training compute than next-chunk reasoning RL to get there. The authors report this advantage held consistently across both in-domain mathematical reasoning tasks and out-of-domain reasoning tasks.

The paper also reports a separate finding with implications for how these methods should be evaluated: higher accuracy measured before the RLVR stage does not necessarily carry over into higher accuracy after RLVR. The authors argue this means no-CoT training strategies should be evaluated within the context of the full post-training pipeline, not just at an earlier stage. The abstract does not name a specific benchmark, dataset or base model used in the comparison, does not give absolute accuracy figures for either method, and does not include code, dataset or model release details.

Key facts

  • Mixed SFT, a single supervised fine-tuning stage trained jointly on no-CoT and long-CoT data, reaches a clearly higher post-RLVR performance ceiling than next-chunk reasoning RL.
  • Mixed SFT needs over 60 times less training compute than next-chunk reasoning RL to reach that higher ceiling.
  • The advantage holds consistently across in-domain mathematical reasoning tasks and out-of-domain reasoning tasks.
  • Higher accuracy before the RLVR stage does not necessarily translate into higher accuracy after RLVR.
  • The paper, by Yinhao Tang and ten co-authors, was submitted to arXiv on August 24, 2026, and does not disclose the benchmark, dataset or base model used.

Why it matters

Post-training pipelines increasingly lean on next-chunk reasoning RL to extract value from no-CoT data such as worked solutions and textbook derivations, which carry reasoning-rich content without explicit chain-of-thought annotations. But prior evaluations of that method mostly compared it against plain SFT, leaving it unclear whether its gains came from the RL formulation itself or simply from better exposure to no-CoT data. This study isolates that question directly and finds that a much simpler alternative, Mixed SFT, reaches a clearly higher post-RLVR ceiling than next-chunk reasoning RL while using over 60 times less training compute. That result challenges the assumption that an RL stage is necessary to get value out of no-CoT corpora.

Who it affects

Teams building or post-training reasoning-capable language models that draw on no-CoT sources such as worked solutions and textbook material, and that are weighing an RL-based training stage against a supervised one. It is most relevant to groups running RLVR pipelines who need to decide how to spend limited training compute on the stage that comes before RLVR.

How to use it

The abstract gives no code, dataset or model release, so there is nothing to run yet. The practical implication is procedural: instead of training an RL stage around next-chunk prediction rewards, run a single supervised fine-tuning pass that mixes no-CoT and long-CoT data together, then proceed into RLVR as usual. Because the paper does not name the benchmark or base model it used, reproducing the exact recipe would require testing it against a team's own setup.

How solid is it

The comparison is framed by the authors as a controlled study, run across both in-domain mathematical reasoning and out-of-domain reasoning tasks, and it directly targets a specific gap in prior work: that next-chunk reasoning RL had previously been benchmarked against conventional SFT rather than a matched Mixed SFT baseline. The abstract backs its claim only with the relative compute figure, over 60 times less, and a qualitative "clearly higher ceiling" statement; it gives no absolute accuracy numbers for either method and does not name the benchmark or base model used, which limits what can be checked from the abstract alone.

Risks and caveats

The findings rest on the authors' own abstract, with no accompanying code, dataset or released model to verify them independently. No benchmark, dataset or base model is named, so the exact conditions of the compute and performance comparison are not disclosed. The authors flag their own caveat: higher accuracy before the RLVR stage does not necessarily hold after RLVR, meaning intermediate results can be misleading without evaluating the complete post-training pipeline.

“Mixed SFT achieves a clearly higher post-RLVR performance ceiling than next-chunk reasoning RL while requiring over 60 times less training compute.”

— the paper's abstract