Google's RRSI curbs overfitting in self-improving AI agent harnesses
An LLM agent's capability comes largely from its harness: the prompts, control flow, tooling, memory and context management wrapped around a frozen backbone model. Recent methods increasingly automate the editing of this harness by iteratively proposing and selecting component-wise changes, a form of recursive self-improvement (RSI) applied at the agent-system level rather than to the model itself. The problem the paper addresses is that this kind of evolution can overfit: it memorizes the specific training tasks it is evolved against, producing large gains on those tasks that shrink or even vanish when the harness is tested on out-of-distribution benchmarks.
Regularized Recursive Self-Improvement of Agent Harnesses (RRSI) addresses this by building regularization directly into both stages of the evolution loop. On the proposal side, a temporally annealed budget limits how many edits a single candidate can bundle together, and the proposer is pushed toward unexplored directions based on the history of what has already been tried. On the selection side, a critic screens out proposals that look tailored to one specific benchmark rather than generally useful, while a pruner discards changes that are too small, too costly, or no longer helping. Together, these constraints are meant to keep the harness accumulating mechanisms that generalize, rather than benchmark-specific tricks or noise.
Across eight benchmarks spanning coding, agentic workspace and engineering design tasks, RRSI gains up to 14.1 points on the split it is evolved against (its in-distribution performance) and up to 4.7 points on the five benchmarks held out as out-of-distribution tests. The harness RRSI produces also runs on 30% fewer policy tokens than the unregularized evolution baseline it is compared against, meaning it reaches its results more cheaply per run as well as more robustly.
The code is public at github.com/google-research/rrsi, with a project page at regularized-rsi.com.
Key facts
- Agent harnesses (prompts, control flow, tools, memory, context management) are increasingly evolved automatically, and this recursive self-improvement can overfit the training tasks it runs on.
- RRSI regularizes both proposal (a temporally annealed edit budget plus a push toward unexplored trajectories) and selection (a critic and a pruner) in the evolution loop.
- Across eight benchmarks spanning coding, agentic workspace and engineering design, RRSI gains up to 14.1 points in-distribution and up to 4.7 points on five out-of-distribution benchmarks.
- The RRSI-produced harness runs on 30% fewer policy tokens than the unregularized evolution baseline.
- Code is published at github.com/google-research/rrsi, under Google's research GitHub organization, with a project page at regularized-rsi.com.
Why it matters
Agent harnesses are increasingly optimized by automated evolution rather than by hand, which makes the evolution process itself a target worth getting right. The paper's core finding is that unconstrained recursive self-improvement can quietly memorize the training tasks it is evolved against: a harness that looks much stronger on its training benchmark can lose most or all of that edge on a different benchmark. RRSI's contribution is a set of constraints on the propose-and-select loop meant to keep those gains from evaporating outside the training distribution.
Who it affects
Teams building agent harnesses for coding, agentic workflows or engineering design who want to improve them through automated evolution rather than manual tuning, and researchers evaluating recursive self-improvement methods where strong in-distribution numbers alone can be misleading.
How to use it
The method's code is published at github.com/google-research/rrsi, with a project page at regularized-rsi.com. The regularization scheme, an annealed edit budget with novelty-seeking on the proposal side and a critic plus pruner on the selection side, is described as a constraint that can be applied to a harness-evolution loop rather than a full replacement architecture.
How solid is it
The results come from evaluation across eight benchmarks covering coding, agentic workspace and engineering design tasks, with five of those held out specifically as out-of-distribution tests to check for overfitting rather than just reporting in-distribution scores. The code is public at the GitHub link above, under Google's research organization, google-research, which allows the reported numbers to be checked directly. The source text does not name the paper's individual authors or their institutions beyond that GitHub organization, does not name the eight benchmarks individually, and does not identify the 'unregularized evolution' baseline by name.
Risks and caveats
The headline gains (up to 14.1 points in-distribution, up to 4.7 points out-of-distribution) are stated as maximums, not averages, so typical improvement across the full set of benchmarks is likely smaller. Both figures are given only as points on an unspecified accuracy metric, so they should not be read as percentages or compared directly across benchmarks. The 30% token reduction is measured against 'the unregularized evolution' baseline, which the source does not further identify by name.