ProgramGym offers 1,648 training environments for coding agents

The paper starts from a shift in how coding agents are used. They are increasingly asked to build programs from scratch, as software engineers do, rather than to resolve issues inside an existing repository or to generate a repository from a requirements document. The authors say two things stand in the way. Scalable, verifiable training environments that force an agent to discover what is required and to decide how the system should be designed remain scarce. And existing agent workflows cannot sustain long-horizon development in which the agent explores the reference program before implementing anything.

To address both, the authors introduce ProgramGym, a framework made of two parts. Program-Env constructs verifiable training environments from repositories, using a feedback-driven loop that repairs runtime sandbox builds and the quality of black-box tests. With this pipeline the authors obtain 1,648 training environments spanning C, C++, Go and Rust. Program-Agent is a workflow that combines role-isolated stages of analysis, implementation and review with revision management, which handles compilation checks and failure recovery.

The experiments run on ProgramBench, and the authors report that both components improve performance on program reproduction. Fine-tuning Qwen3.8-27B on trajectories from the constructed environments raises its performance from 34.87% to 42.84%. Program-Agent then raises the base and SFT models to 55.71% and 56.25%, and the authors say both figures exceed some flagship models. The abstract does not name those models or give their scores.

The authors say they will release ProgramGym to support further research on coding agents that build programs from scratch. The release is a promise at this point; the abstract gives no date, licence or repository link.

Key facts

ProgramGym has two parts: Program-Env, which builds verifiable training environments from repositories, and Program-Agent, a role-isolated workflow of analysis, implementation and review with revision management.
Program-Env produced 1,648 training environments spanning C, C++, Go and Rust, using a feedback-driven loop that repairs sandbox builds and black-box test quality.
On ProgramBench, fine-tuning Qwen3.8-27B on trajectories from these environments raises performance from 34.87% to 42.84%.
Program-Agent lifts the base and SFT models to 55.71% and 56.25%, which the authors say exceeds some unnamed flagship models.
The authors say they will release ProgramGym; no date, licence or link is given.

Why it matters

Building a program from scratch is a different task from fixing an issue in a repository or writing code from a finished specification. The agent has to explore a reference program, work out what is required and decide how the system should be designed. The authors argue that verifiable training environments for this kind of work are scarce, and ProgramGym is their attempt to supply them at scale. The reported gain from fine-tuning Qwen3.8-27B is 7.97 points, from 34.87% to 42.84% on ProgramBench.

Who it affects

The work is aimed at researchers and teams training or evaluating coding agents that build programs from scratch. The environments cover C, C++, Go and Rust, so the training data leans toward systems-level languages. Teams that build agent workflows may also look at Program-Agent's split into analysis, implementation and review stages.

How to use it

There is nothing to download yet. The authors state that they will release ProgramGym, but the abstract gives no release date, licence or repository link. Until then the paper is a description of the approach: build verifiable environments from repositories with Program-Env, fine-tune on the resulting trajectories, and run the model inside Program-Agent's role-isolated workflow.

How solid is it

The claims come from the paper's abstract and are the authors' own results on ProgramBench. The abstract names no authors or institutions. The metric behind the percentages is not named; only "performance" on ProgramBench is stated. The flagship models that the results are said to exceed are not named, and no scores for them are given. The abstract does not state how many source repositories were used, how the 1,648 environments split across the four languages, or what generated the SFT trajectories. The gap between the two Program-Agent results is small: 56.25% for the SFT model against 55.71% for the base model, or 0.54 points.

Risks and caveats

The release is only promised, so the environments cannot be checked independently yet. Without the metric definition and the names of the flagship models, the comparison to them cannot be judged. It is also not stated whether the 55.71% and 56.25% figures belong to the same Qwen3.8-27B model as the 34.87% to 42.84% result, though the abstract presents them right after it. All numbers are from a single benchmark, ProgramBench.