Glonce

PaperGym trains Qwen3 models to plan research using paper rubrics

researchThe Glonce editorsHugging Face🔥56Glonce rating56Impact65Novelty75Relevance95Surprise50Players40Credibility85
PaperGym trains Qwen3 models to plan research using paper rubrics

Research planning, the authors argue, is the decisive capability for AI systems meant to do science, but it resists standard reinforcement learning: a research plan has no single verifiable answer, so RL lacks the critic it needs. Prior pipelines tried to supply that critic with rubrics pulled from scientific papers, but they drew both the question and the grading criteria from the same content, so a model could earn reward by paraphrasing the source rather than actually solving the planning task. The researchers introduce PaperGym, a framework that avoids this by splitting a paper in two: the task is synthesized from its research goal and background, while the grading criteria, covering methodological innovation and experimental design, are derived separately from its method and experiments. This separation cuts criterion leakage to 3.7%, against a range of 11.90% to 34.10% in existing datasets.

The rubric is then used twice in training. It first serves as privileged context for a self-teacher called OPSD, then becomes the reward signal for a reinforcement-learning stage called GRPO. Run in that order, on Qwen3 models at 1.7B, 4B, and 8B parameters, this two-stage schedule beats plain supervised fine-tuning, either stage used alone, and the reverse ordering of the two stages, lifting the average across five benchmarks by +5.6, +5.0, and +4.8 points for the three model sizes respectively.

With the training recipe held fixed, models trained on the newly built PaperGym-20k corpus win 58.1% of three-way comparisons, against 28.2% for models trained on the existing RubricHub Science dataset. The trained Qwen3-8B reaches a score of 73.48 on the ResearchQA benchmark, which the authors say is higher than the far larger Kimi K2.6 achieves, though Kimi K2.6's own score is not given in the source.

Alongside the paper, the team releases the PaperGym pipeline itself, a 20,000-instance training corpus called PaperGym-20k, and two new benchmarks, PaperGym-Innov and PaperGym-Design.

Key facts

  • PaperGym builds each training task from a paper's goal and background while deriving the grading rubric separately from its method and experiments, cutting criterion leakage to 3.7% versus 11.90% to 34.10% in existing datasets.
  • The rubric is used twice: first as privileged context for a self-teacher called OPSD, then as the reward for a reinforcement-learning stage called GRPO.
  • Trained on Qwen3-1.7B, 4B, and 8B, the OPSD-then-GRPO schedule beats supervised fine-tuning, either stage alone, and the reverse order, raising five-benchmark averages by +5.6, +5.0, and +4.8 points respectively.
  • With the recipe held fixed, models trained on the new PaperGym-20k corpus win 58.1% of three-way comparisons against 28.2% for the existing RubricHub Science dataset.
  • The trained Qwen3-8B scores 73.48 on ResearchQA, said to exceed the much larger Kimi K2.6 (whose own score is not disclosed); the team releases the pipeline, the 20,000-instance PaperGym-20k corpus, and the PaperGym-Innov and PaperGym-Design benchmarks.

Why it matters

Research planning has no single correct answer, so it has been hard to give reinforcement learning the graded feedback it needs to improve at it. PaperGym's fix is structural rather than a bigger model: split a paper so the task and the grading rubric come from different sections, which stops a model from scoring well simply by echoing the source it is meant to be reasoning about.

Who it affects

The work targets teams building AI systems meant to plan scientific research, and reinforcement-learning researchers designing reward signals for open-ended tasks. The experiments are run on the Qwen3 family at three sizes, 1.7B, 4B, and 8B parameters, and are compared against RubricHub Science, an existing rubric dataset, and against Kimi K2.6, a larger model.

How to use it

The authors release the PaperGym pipeline for turning papers into training environments, the resulting PaperGym-20k corpus of 20,000 instances, and two evaluation sets, PaperGym-Innov and PaperGym-Design. The source gives no pricing, license terms, or release date for any of these.

How solid is it

The central comparisons are quantitative and relative: a leakage rate of 3.7% against 11.90% to 34.10% for existing pipelines, five-benchmark average gains of +5.6, +5.0, and +4.8 points for the three model sizes, Qwen3-1.7B, 4B, and 8B, each measured against plain fine-tuning, either training stage alone, and the reverse stage order, and a 58.1% versus 28.2% win rate against RubricHub Science with the recipe held fixed. The Qwen3-8B result of 73.48 on ResearchQA is presented as beating Kimi K2.6, but the source does not state Kimi K2.6's actual score, so the size of that gap cannot be checked from the text alone.

Risks and caveats

The source text names no authors or institutions and gives no publication or release date, so provenance cannot be verified here. The acronyms OPSD and GRPO are used without being spelled out, and the five benchmarks behind the headline averages, along with ResearchQA itself, are not described. All comparative claims come from the authors' own reporting of their own method against baselines they selected.