Glonce

LEGO-RL lifts SWE-bench Verified scores on Claude Code, OpenCode and OpenHands SDK

researchThe Glonce editorsHugging Face🔥71Glonce rating71Impact65Novelty75Relevance95Surprise45Players30Credibility85
LEGO-RL lifts SWE-bench Verified scores on Claude Code, OpenCode and OpenHands SDK

Yiming Du and co-authors present LEGO-RL, a framework for training coding agents with reinforcement learning directly inside the long-running harnesses that already manage their tool calls, repository context and execution feedback. The paper's starting problem is that these native harness environments are a poor fit for policy-gradient training: environmental crashes and reward hacking corrupt the outcome signal the trainer relies on, and gaps between what happens during a rollout and what the trainer later sees decouple rollout behavior from the policy updates meant to follow it.

LEGO-RL is built to close that gap without modifying a harness's internal control flow, resting on three pillars. The first, faithful optimization, uses an in-process LLM proxy that captures the raw generation stream, so token-level alignment and the trainer's recomputation of log-probabilities stay accurate even when the harness compacts or re-serializes its own data. The second, reliable execution, adds scalable sandbox orchestration with image caching and stage-wise defenses meant to blunt reward hacking. The third, observable training, pairs an integrated plugin that automates validation and monitoring with a Live UI for inspecting individual training trajectories in detail.

The authors evaluate LEGO-RL by training the sparse mixture-of-experts model Qwen3.5-35B-A3B with the GSPO algorithm inside three unmodified coding-agent harnesses, measuring SWE-bench Verified scores before and after. On OpenHands SDK the score rose from 64.0% to 70.4%, a gain of 6.4 percentage points. On Claude Code it rose from 62.4% to 68.2%, a gain of 5.8 percentage points. On OpenCode it rose from 57.2% to 66.6%, the largest gain at 9.4 percentage points. Across all three, the authors report that the correlation between rollout probabilities and training probabilities stayed above 0.99 throughout, which they offer as evidence that the harness's view of the policy and the trainer's view of it remained aligned during training.

The paper gives no comparison against other RL-for-coding-agent frameworks, so there is no baseline in the text to show how large this advantage is against existing approaches. It also states no training cost, compute budget or wall-clock duration, and says nothing about whether LEGO-RL's code, the trained model, or any other artifact will be released.

Key facts

  • LEGO-RL, from Yiming Du and co-authors, lets policy-gradient reinforcement learning run inside existing coding-agent harnesses without changing their internal control flow.
  • It rests on three pillars: in-process LLM proxying for faithful token-level optimization, scalable sandbox orchestration with image caching and stage-wise defenses against reward hacking, and an integrated monitoring plugin paired with a Live UI for trajectory diagnostics.
  • Trained with LEGO-RL and the GSPO algorithm, the sparse MoE model Qwen3.5-35B-A3B raised its SWE-bench Verified score from 64.0% to 70.4% on OpenHands SDK, 62.4% to 68.2% on Claude Code, and 57.2% to 66.6% on OpenCode.
  • Across all three harnesses, the rollout-training probability correlation stayed above 0.99 throughout training, the authors report.
  • The paper does not compare LEGO-RL against other RL-for-coding-agent frameworks, and discloses no training cost, compute budget, or plan to release code or models.

Why it matters

Reinforcement learning for coding agents increasingly happens inside the same long-running harnesses that manage an agent's tool calls, repository context and execution feedback, rather than in a separate, simplified training environment. The authors argue that setup is a poor fit for policy-gradient training on its own: environmental crashes and reward hacking corrupt the signal the trainer uses to judge outcomes, and gaps between what a harness does during a rollout and what the trainer later sees decouple the agent's rollout behavior from the policy updates meant to follow it. LEGO-RL is presented as a way to close that gap without rewriting a harness's internal control flow, which matters because it lets RL be layered onto agent tooling teams already run, rather than forcing a move to a separate, purpose-built training setup.

Who it affects

The immediate audience is ML researchers and infrastructure engineers who train coding agents with reinforcement learning on top of existing harnesses instead of custom-built training environments. The paper specifically targets teams already using OpenHands SDK, Claude Code or OpenCode as the execution layer for their agents, since LEGO-RL is designed to plug into those three without altering how they work internally. It is also relevant to anyone evaluating Qwen models for coding-agent work, since the reported gains were measured specifically on the sparse mixture-of-experts model Qwen3.5-35B-A3B.

How to use it

LEGO-RL is not a single technique but three components working together. An in-process LLM proxy captures the raw generation stream during rollouts, which keeps token-level alignment and the trainer's recomputation of log-probabilities accurate even when the harness compacts or re-serializes its own data. A sandbox-orchestration layer handles execution at scale, using image caching plus stage-wise defenses aimed at reward hacking. An integrated plugin automates validation and monitoring, feeding a Live UI built for inspecting individual training trajectories. The authors apply this setup by training Qwen3.5-35B-A3B with the GSPO algorithm inside three coding-agent harnesses, OpenHands SDK, Claude Code and OpenCode, without changing any of the three harnesses' internal control flow. The paper does not say whether LEGO-RL's own code will be released, so there is currently no stated way for other teams to run it themselves.

How solid is it

The evidence is the authors' own before-and-after SWE-bench Verified numbers on a single base model, Qwen3.5-35B-A3B: 64.0% to 70.4% on OpenHands SDK, 62.4% to 68.2% on Claude Code, and 57.2% to 66.6% on OpenCode, gains of 5.8 to 9.4 percentage points depending on the harness. As a secondary check, the authors report the rollout-training probability correlation staying above 0.99 across all three harnesses throughout training, which they offer as evidence that the harness's view of the policy and the trainer's view of it stayed aligned. Beyond crediting Yiming Du and co-authors, the write-up gives no institutional affiliation and no independent or third-party validation of these numbers.

Risks and caveats

All of the reported gains come from the authors' own experiments on one base model, so it is not clear how the results would generalize to other model families or sizes, or to coding-agent harnesses beyond the three tested. Because the paper does not compare LEGO-RL to other RL-for-coding-agent approaches, there is no way to tell from this text how large its advantage is over existing methods, if any. No training cost, compute budget or wall-clock duration is given, and the paper says nothing about releasing the code, the trained model or any other artifact, so external teams currently have no stated path to reproduce or adopt it.

“The native execution environments of these harnesses are inherently misaligned with policy-gradient training: environmental crashes and reward hacking corrupt outcome signals, while train-inference discrepancies decouple rollout behavior from policy updates.”

— the paper's authors