Glonce

Skill-Entropy RL nearly doubles Qwen3-4B-Instruct's score on Skill^2-Bench

researchThe Glonce editorsHugging Face🔥72Glonce rating72Impact68Novelty72Relevance95Surprise52Players42Credibility80
Skill-Entropy RL nearly doubles Qwen3-4B-Instruct's score on Skill^2-Bench

Long-horizon reasoning in current LLMs often forces a model to switch between different kinds of skills inside one reasoning chain, for example doing a math derivation and then using the result to plan a schedule. Yinghui He and co-authors call these cross-skill long-horizon tasks and argue that existing benchmarks test skills one at a time, with no principled way to measure how well a model handles the switch itself. To close that gap, they introduce Skill Entropy, a measure of how difficult it is to move from one skill to another.

Building on that measure, the authors present Skill^2-Bench, a benchmark of cross-skill long-horizon tasks assembled from 558 skills across 9 verifiable and open-ended domains. Every task carries a task-level skill-entropy score and is sorted into one of three difficulty tiers based on it. Running 8 frontier models and 4 open-source models through Skill^2-Bench turns up what the authors call a skill-switching gap: accuracy falls as a task's skill entropy rises, meaning models do worse the more a task forces them to hop between different reasoning skills. The text does not name the specific models tested or state how large the gap is.

The authors then turn skill entropy into a training signal, Skill-Entropy RL: a reinforcement-learning setup where the model has to predict, at every step, both the answer and which skill it used to reach it. The reward combines ordinary step-level correctness with a skill-entropy reward that checks how well the model's predicted sequence of skills matches the task's correct, or gold, skill sequence. Applied to two open Qwen3 models, the method raises Qwen3-4B-Instruct's Skill^2-Bench score from 34.4% to 68.4%, nearly double and a 34.0-point gain, and Qwen3-1.7B's from 14.6% to 40.1%, about 2.7 times and a 25.5-point gain, beating what the paper describes as competitive baselines in both cases, though it gives no baseline scores.

The authors report that the same pipeline can also be applied to existing training data rather than only to Skill^2-Bench-specific tasks, pointing to OpenR1-Math as an example; they present this as evidence that skill entropy is a reusable training signal rather than a benchmark-only trick. Code for Skill-Entropy RL is public at github.com/Gen-Verse/Skill-Entropy-RL. The text gives no institution, publication venue, or date for the study.

Key facts

  • Skill Entropy is a new metric for how hard it is for an LLM to switch from one reasoning skill to another inside a multi-step task.
  • Skill^2-Bench, the benchmark built to test it, spans 558 skills across 9 verifiable and open-ended domains, with tasks sorted into three difficulty tiers by their skill-entropy score.
  • Testing 8 frontier models and 4 open-source models on Skill^2-Bench found a skill-switching gap: accuracy drops as a task's skill entropy rises.
  • Skill-Entropy RL, which rewards a model for predicting the right skill at each step alongside the right answer, raises Qwen3-4B-Instruct's score from 34.4% to 68.4% (nearly double) and Qwen3-1.7B's from 14.6% to 40.1% (about 2.7 times).
  • The same training pipeline can also be applied to existing data such as OpenR1-Math, and the code is public at github.com/Gen-Verse/Skill-Entropy-RL.

Why it matters

Multi-step AI reasoning increasingly needs a model to switch between different kinds of skills inside one chain, for example working out a math result and then using it to plan a schedule. The authors call these cross-skill long-horizon tasks and argue that existing benchmarks test skills one at a time, with no principled way to measure how well a model handles the switch itself. Skill Entropy is proposed as that missing measure, a score for how hard it is to move from one skill to another. Testing it across 12 models turns up a skill-switching gap: accuracy drops as a task demands harder skill transitions, concrete evidence that skill-switching, not any single skill in isolation, is a distinct weak point in current LLM reasoning.

Who it affects

The work is aimed at people building and evaluating LLMs for multi-step and agentic reasoning that mixes skill types, such as combining calculation, planning and open-ended writing in one chain. Skill^2-Bench evaluates 8 unnamed frontier models and 4 unnamed open-source models, so any lab shipping a reasoning-focused model is implicitly being measured by it. The demonstrated training gains are on two smaller open models, Qwen3-4B-Instruct and Qwen3-1.7B, which matters most to teams fine-tuning smaller models rather than relying on the largest frontier systems.

How to use it

The code for Skill-Entropy RL is public at github.com/Gen-Verse/Skill-Entropy-RL. In this setup, a model is trained to output, at each reasoning step, both an answer and the skill it used to reach it; the RL reward then combines ordinary step-level correctness with a skill-entropy reward that scores how well the model's predicted skill sequence matches the task's gold skill sequence. The authors report that the same pipeline can be run on existing datasets such as OpenR1-Math, not only on Skill^2-Bench-specific data, which is what lets them call skill entropy a reusable training signal rather than a benchmark-only trick.

How solid is it

The results rest on a purpose-built benchmark, 558 skills across 9 domains, with every task scored for skill entropy and split into three difficulty tiers, tested across 12 models for the base benchmark result and two Qwen3 models for the training result. The training gains are large for both model sizes: Qwen3-4B-Instruct rises from 34.4% to 68.4% on Skill^2-Bench, nearly double and a 34.0-point gain; Qwen3-1.7B rises from 14.6% to 40.1%, about 2.7 times and a 25.5-point gain. Both beat what the paper calls competitive baselines, though it gives no baseline scores. The text also does not state how large the skill-switching gap itself is, only that accuracy decreases as entropy rises, so that particular finding cannot be sized from what is given.

Risks and caveats

The text does not identify which specific models make up the '8 frontier and 4 open-source' count evaluated on the base benchmark, nor does it quantify the skill-switching gap itself or give scores for the 'competitive baselines' that Skill-Entropy RL is said to outperform. It also gives no institution, publication venue, or date for the study; those details would require the full paper rather than the abstract alone.