PILOT gives long-horizon AI agents live self-improvement
Most self-improvement methods for AI agents only process what happened after a run finishes, so they cannot redirect a task while it is still going or immediately apply a lesson the run just produced. The authors of a new paper argue self-improvement should instead be live: emerging experience should be used both to redirect the active run and to update the persistent harness around it. They note that existing architectures do not fully support this. Single-agent self-correction mixes task execution and self-assessment inside one context, while subagent delegation separates execution from oversight but typically cannot redirect a subagent once it is already running.
The paper presents PILOT, a supervisor-worker harness built on two coupled mechanisms. Live steering lets a separate supervisor redirect or abort the active worker while it is still executing. Live self-evolution distills the procedures and failure modes that surface during execution into reusable skills and memory, so the lesson is available immediately rather than only in a later run.
PILOT was evaluated across two frozen model backbones and three benchmarks, and it ranked first in five of the six resulting configurations (the specific benchmarks and the rival harnesses it beat are not named in the text). On Terminal-Bench 2.0, it outperformed counterpart harnesses by up to 9.8 percentage points. In the self-improvement setting specifically, PILOT gained 14.6 points with the GLM-5.1 backbone and 12.4 points with Kimi-K2.6. Alongside those gains, mean output tokens fell by 42.9% and 47.4%, while successful evaluations per million output tokens rose by 110.3% and 134.0%, respectively.
Key facts
- PILOT is a supervisor-worker harness combining live steering, where a supervisor can redirect or abort the active worker mid-run, with live self-evolution, which distills procedures and failure modes into reusable skills and memory during the run.
- Evaluated across two frozen backbones and three benchmarks, PILOT ranked first in five of six configurations.
- On Terminal-Bench 2.0, PILOT outperformed counterpart harnesses by up to 9.8 percentage points.
- In the self-improvement setting, PILOT gained 14.6 points with the GLM-5.1 backbone and 12.4 points with Kimi-K2.6.
- Mean output tokens fell by 42.9% and 47.4%, while successful evaluations per million output tokens rose by 110.3% and 134.0%, respectively.
Why it matters
Existing self-improvement schemes act only after a run ends, so a mistake made early in a long task cannot be caught and corrected before the task finishes. PILOT's authors argue the fix has to be live: the harness needs to redirect the active run and update itself from the same experience the run is generating, not just log it for next time. Single-agent self-correction cannot do this cleanly because it mixes doing the task with judging it in one context, and standard subagent delegation cannot do it either because a supervisor that spawns a subagent typically has no way to steer it once it is running.
Who it affects
The paper targets builders of long-horizon AI agents, the kind that run extended, multi-step tasks such as terminal or coding sessions where a wrong turn early on compounds. It was tested against two backbone models, GLM-5.1 and Kimi-K2.6, so it is directly relevant to teams building or evaluating agent harnesses on top of those or similar models.
How to use it
PILOT works through two coupled mechanisms rather than a single trick. Live steering gives a separate supervisor the ability to redirect or abort the worker while it is still executing, instead of waiting for it to finish. Live self-evolution turns procedures and failure modes surfaced mid-run into reusable skills and memory that the harness can draw on immediately, within the same run or the next one. The text does not mention any code, data or model release, or any pricing or licensing terms.
How solid is it
The evaluation spans two frozen backbones and three benchmarks, six configurations in total, and PILOT ranks first in five of them. On Terminal-Bench 2.0 specifically it beats unnamed counterpart harnesses by up to 9.8 percentage points. In the self-improvement setting it gains 14.6 points with GLM-5.1 and 12.4 points with Kimi-K2.6, while also cutting mean output tokens by 42.9% and 47.4% and raising successful evaluations per million output tokens by 110.3% and 134.0%, respectively.
Risks and caveats
The source text names neither individual authors nor an affiliated institution, and carries no publication or submission date. It does not name the three benchmarks used beyond Terminal-Bench 2.0, nor the counterpart harnesses PILOT is compared against. The text also does not explicitly restate which backbone each of the token-reduction and evaluation-efficiency percentages belongs to; the pairing with GLM-5.1 and Kimi-K2.6 follows only from the order the figures are given in. No code, data or model release is mentioned.
“We argue that self-improvement should instead be live, using emerging experience both to redirect the active run and to update the persistent harness.”
— the paper's authors