Glonce

Study: only 37 of 1,598 general RL methods reached training frameworks

researchThe Glonce editorsHugging Face🔥58Glonce rating58Impact45Novelty55Relevance90Surprise40Players15Credibility75
Study: only 37 of 1,598 general RL methods reached training frameworks

Practitioners apply reinforcement learning methods to language models through training frameworks, so a method that a framework includes is one configuration change away for its users. A new paper measures how often that actually happens.

The authors trace seven open-source frameworks for RL post-training through their commit histories and link them to 6,939 screened arXiv papers published from January 2025 to June 2026. Of 1,598 papers that propose a general RL training method, 37 reached at least one framework.

The pace of inclusion did not follow the pace of publishing. Across five quarterly cohorts of papers, the number included within 180 days stayed between 5 and 7, while the cohorts grew from 72 to 386 papers.

Included papers differ from the rest. Compared with papers from the same quarters, they more often trained policies of 30B parameters or more (41 against 10 percent) and more often changed a coefficient, clip, mask or weight in the existing loss (29 against 9 percent). Both gaps hold when papers from the companies that maintain the frameworks are excluded. Code release was equally common in both groups.

The frameworks' issues and pull requests show that few methods are turned away in public. More than nine in ten of the general methods no framework included have no thread there that cites or names them, and many proposals that do appear stay open without a reply.

Two further records, the frameworks' issues and the technical reports of 13 large open-weight models, show that users rarely report training failures in public and that the reports state 8 of the 13 widely included methods.

The authors release the data as a tracker that links training failures to framework implementations and lists each framework's contribution rules and earlier proposals.

Key facts

  • Of 1,598 arXiv papers proposing a general RL training method, 37 reached at least one of seven open-source RL post-training frameworks.
  • Across five quarterly cohorts, papers included within 180 days stayed between 5 and 7 while the cohorts grew from 72 to 386 papers.
  • Included papers more often trained policies of 30B parameters or more (41 against 10 percent) and more often changed a coefficient, clip, mask or weight in the existing loss (29 against 9 percent).
  • More than nine in ten of the general methods no framework included have no thread in the frameworks' issues and pull requests that cites or names them.
  • The authors release a tracker linking training failures to framework implementations, with each framework's contribution rules and earlier proposals.

Why it matters

Frameworks are how most practitioners reach new RL methods for language models: once a framework includes a method, using it is one configuration change away. This study puts a number on how narrow that path is. Only 37 of 1,598 general-method papers got through, and the count of papers included within 180 days held at 5 to 7 per quarterly cohort while cohorts grew from 72 to 386 papers. Publishing volume is rising far faster than what ends up in usable code.

Who it affects

Practitioners who train language models with open-source RL post-training frameworks, since their menu of methods is set by what the frameworks include. It also affects researchers who propose new RL methods and want them used, and the maintainers of the frameworks, whose issues and pull requests the study examined. The authors note that included papers more often trained policies of 30B parameters or more, so work at that scale is better represented.

How to use it

The authors release the data as a tracker that links training failures to framework implementations and lists each framework's contribution rules and earlier proposals. For a researcher, that gives a place to check what a framework has already seen before proposing a method. For a practitioner, it is a way to see which methods a framework actually carries. The study's own pattern is a guide too: included papers more often changed a coefficient, clip, mask or weight in the existing loss (29 against 9 percent).

How solid is it

The numbers come from commit histories of seven frameworks linked to 6,939 screened arXiv papers over January 2025 to June 2026, plus the frameworks' issues and pull requests and the technical reports of 13 large open-weight models. The authors check that the two gaps hold without papers from the companies that maintain the frameworks. The abstract does not name the seven frameworks. It also gives no per-framework breakdown of inclusions. The account here rests on the paper's abstract.

Risks and caveats

The study measures association, not cause. The abstract does not say why so few papers are included or give a causal mechanism for the gaps. Public threads are only part of what happens: the authors say few methods are turned away in public, and that users rarely report training failures in public, so the public record is thin on both rejection and failure. The inclusion window is 180 days, so later inclusion is not counted in the cohort figures. The abstract does not name the 13 open-weight models or which 13 methods are widely included.

“so a method that a framework includes is one configuration change away for its users”

— From the paper's abstract