Glonce

ClawGym II lifts agent Pass@1 by up to 14.81 points with black-box RL

researchThe Glonce editorsHugging Face🔥57Glonce rating57Impact65Novelty78Relevance98Surprise45Players50Credibility82
ClawGym II lifts agent Pass@1 by up to 14.81 points with black-box RL

Researchers describe ClawGym II, a unified black-box RL framework built to train general AI agents through complex agent harnesses, the orchestration layers that coordinate an agent's interaction with its environment on long-horizon tasks. The paper argues that reinforcement learning through such harnesses has remained largely unexplored, because scaling training to long-horizon agent tasks runs into fundamental difficulties. The framework has several parts. A sandbox-based execution infrastructure isolates task environments and harnesses inside temporary sandboxes, allowing large numbers of rollouts to run concurrently. A serving proxy sits at the model boundary and captures model calls, which decouples policy optimization from the opaque internals of the harness itself. The captured calls are organized into prefix trees to reconstruct multi-turn trajectories and make training more efficient, and the team adapts both critic-based PPO and critic-free GRPO to optimize over that recovered tree structure. The framework also maintains training-inference consistency throughout optimization, and adds mix-harness training, letting a single model be jointly optimized across multiple different harnesses at once. Tested with the Qwen3-30A3B model, black-box RL improved Pass@1 on the ClawGym-Bench benchmark by 9.98 points when training ran through the OpenClaw harness and by 14.81 points when it ran through Claude Code, while staying stable over 200 to 400 optimization steps. The authors also report consistent gains on harder tasks, JobBench and OfficeQA, though the source text gives no specific numbers for those results.

Key facts

  • ClawGym II is a black-box RL framework for training general AI agents through complex, opaque agent harnesses such as OpenClaw and Claude Code.
  • With the Qwen3-30A3B model, it improves Pass@1 on ClawGym-Bench by 9.98 points via OpenClaw and 14.81 points via Claude Code.
  • Training stays stable over 200 to 400 optimization steps.
  • Key components: sandboxed concurrent rollouts, a serving proxy that decouples policy optimization from harness execution, prefix-tree reconstruction of trajectories, and adapted PPO/GRPO training.
  • Mix-harness training lets one model be jointly optimized across heterogeneous harnesses, and the framework also reports gains on JobBench and OfficeQA without stated figures.

Why it matters

Agent harnesses, the orchestration code that manages an agent's back-and-forth with its environment, have driven big gains on long-horizon tasks, but training agents with reinforcement learning through those harnesses had stayed largely unexplored because scaling it up runs into fundamental problems. ClawGym II is presented as a way to make that training stable and scalable rather than an ad hoc, one-off exercise.

Who it affects

The work targets researchers and engineers building RL training pipelines for agents that operate through complex, third-party or opaque harnesses, including named systems like OpenClaw and Claude Code, rather than agents trained in isolation from their tooling.

How to use it

The framework runs task environments and harnesses inside temporary, isolated sandboxes for large-scale concurrent rollouts. A serving proxy placed at the model boundary captures model calls so policy optimization does not need visibility into the harness internals. Captured calls are reorganized into prefix trees to rebuild multi-turn trajectories, and both critic-based PPO and critic-free GRPO are adapted to train over that tree structure, with training-inference consistency maintained throughout. Mix-harness training extends this to jointly optimizing one model across several different harnesses at once. The source text gives no information on code availability, licensing or pricing.

How solid is it

The framework is tested on the Qwen3-30A3B model, where it raises Pass@1 on the ClawGym-Bench benchmark by 9.98 points through the OpenClaw harness and by 14.81 points through Claude Code, with training reported stable over 200 to 400 optimization steps. The authors say the framework also produces consistent gains on the more challenging JobBench and OfficeQA tasks, but the text does not give numeric results for those two.

Risks and caveats

The source text does not define whether a 'point' of Pass@1 improvement means a percentage point or some other unit, so the size of the gain should be read with that ambiguity in mind. No author names or institutional affiliations, no publication date, and no concrete figures for the JobBench and OfficeQA gains are given in the text.