ReHarness adds backward revision to LM agent memory for continual learning
A paper posted on Hugging Face Papers proposes ReHarness, a recurrent harness framework for agentic continual learning. The setting is language model agents that work over long interaction streams and have to adapt online as the distribution of tasks shifts.
The authors say existing agent harnesses typically fall into one of two camps. Either they evolve their state only through forward information flow, or they rely on extra replay or validation data to optimise the harness. ReHarness is meant to avoid both. It keeps the usual forward read and write, and adds backward revision, all under a strictly online streaming protocol.
The mechanism is framed as a question: what information could have been captured differently in the harness state to better inform the current instance in the stream? After each instance, the resulting trajectory is used as feedback to revise the reflections that were retrieved for that instance. Feedback is thereby propagated backward through the textual harness state, so the stored experience is corrected after the fact rather than only appended to.
Across three continual learning benchmarks and three language model backbones, the authors report that ReHarness consistently outperforms strong raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches. Their further analysis, as described in the abstract, shows that it produces source-grounded and meaningful revisions, corrects overgeneralized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning.
Key facts
- ReHarness is a recurrent harness framework for agentic continual learning that adds backward revision to the usual forward read and write.
- After each instance in an online stream, the resulting trajectory is used as feedback to revise the reflections retrieved for that instance.
- It runs under a strictly online streaming protocol, without extra replay or validation data for harness optimization.
- The authors report consistent gains over raw-context, memory- and reflection-based, and prompt optimization baselines across three benchmarks and three language model backbones.
- Their analysis says it corrects overgeneralized or outdated experience under distribution shifts and improves both stability and plasticity.
Why it matters
Agents that run for long stretches meet shifting conditions, and whatever they stored earlier can go stale or turn out to be too general. The paper's point is that most harnesses only move information forward, writing new reflections as they go. ReHarness lets a finished trajectory reach back and correct the reflection that was used for it, which targets the problem of overgeneralized or outdated experience directly. It also does this without replay or validation data, which the authors name as a dependency of other harness optimization methods.
Who it affects
The work is aimed at people building language model agents that operate over long interaction streams and need to adapt online, and at researchers working on continual learning, agent memory and reflection-based methods. The paper compares against raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches, so those lines of work are its direct reference points.
How to use it
The idea is straightforward to describe: keep a textual harness state of reflections, retrieve reflections for each new instance, and after the instance finishes, use the trajectory as feedback to revise the reflections that were retrieved. No code, model or dataset release is mentioned, so there is nothing to download or run from the source text; anyone wanting to try it would be working from the paper's description.
How solid is it
This is a preprint abstract, and every result in it is the authors' own report. The evaluation covers three continual learning benchmarks and three language model backbones, and the claim is consistent outperformance over the compared baselines. The abstract names no authors or institutions, does not name the three benchmarks or the three backbones, and gives no numeric results such as accuracy or margins of improvement, only the qualitative claim.
Risks and caveats
Without numbers it is not possible to judge how large the gains are. The abstract does not state the computational cost or latency of the backward revision step, which adds work after every instance. The claims about meaningful revisions and improved stability and plasticity come from the authors' own further analysis. Which benchmarks and backbones were used is not named in the source, so how well the result carries over to other agents or tasks is unclear.
“We ask what information could have been captured differently in the harness state to better inform the current instance in the stream.”
— Abstract of the ReHarness paper