Glonce

ACCRUE continual off-policy RL cuts real-robot training time by 31.0% to 68.5%

researchThe Glonce editorsHugging Face🔥58Glonce rating58Impact45Novelty60Relevance95Surprise30Players10Credibility75
ACCRUE continual off-policy RL cuts real-robot training time by 31.0% to 68.5%

A paper on Hugging Face Papers introduces ACCRUE, a continual off-policy reinforcement learning (RL) framework for robot foundation models (RFMs). The starting point is that RL post-training can push robot foundation models past what their demonstrations teach, but real-world deployment asks for more: the robot has to learn new tasks efficiently while keeping the behaviors it already acquired.

The authors say existing approaches each miss half of that. Continual RL methods built on top of vision-language-action models (VLAs) reduce forgetting, but they rely on on-policy updates and offer limited forward transfer, meaning little benefit carries over to the next task. Off-policy methods, by contrast, improve sample efficiency on a single task but do not address how a learner persists across tasks.

ACCRUE combines the two ideas. It carries lightweight actor and critic networks from task to task, so each stage can reuse the behavioral and value knowledge gained through earlier RL. The inherited critic adapts only on transitions from the current task and guides the learning of the current-task actor. To keep earlier behavior intact, the actor is distilled toward cached targets using a small number of retained episodes. Fixed visual features keep the inherited learner and its replay data compatible with updates to the base policy.

The evaluation covers contact-rich assembly sequences, both in simulation and on real-robot hardware. According to the authors, ACCRUE speeds up learning of subsequent tasks with near-zero forgetting, and it significantly reduces real-robot training time by 31.0% to 68.5%.

Key facts

  • ACCRUE is a continual off-policy RL framework for robot foundation models that carries lightweight actor and critic networks across tasks.
  • The inherited critic adapts only on current-task transitions, while the actor is distilled toward cached targets on a small number of retained episodes to preserve prior behavior.
  • Fixed visual features keep the inherited learner and its replay compatible with base-policy updates.
  • Tested on contact-rich assembly sequences in simulation and on real-robot hardware, it accelerates subsequent-task learning with near-zero forgetting.
  • The authors report a 31.0% to 68.5% reduction in real-robot training time.

Why it matters

Robots that learn on the job need to pick up new tasks without losing old ones, and without hours of fresh training each time. The authors frame the gap precisely: on-policy continual RL on top of VLAs limits forward transfer, and off-policy methods help a single task but do not persist across tasks. ACCRUE is pitched as covering both by reusing a small actor and critic from one task to the next. A 31.0% to 68.5% cut in real-robot training time, if it holds up, would matter because real-robot time is the expensive part of RL post-training.

Who it affects

Robotics researchers and teams that post-train robot foundation models or VLAs with RL, especially for contact-rich assembly work where tasks arrive in sequence. It is a research result, so the near-term audience is people building and evaluating continual-learning methods rather than end users of robots.

How to use it

There is nothing to install or run from the information available. No code release, dataset release or publication venue is mentioned in the abstract. A team interested in the idea would reproduce the recipe described: carry a lightweight actor and critic between tasks, adapt the critic only on current-task transitions, distill the actor toward cached targets on a few retained episodes, and keep visual features fixed so the replay stays compatible with base-policy updates.

How solid is it

This is a preprint abstract, and the evidence is the authors' own report of results on contact-rich assembly sequences in simulation and on real-robot hardware. The abstract names no authors or institutions. The baseline against which the 31.0% to 68.5% training-time reduction is measured is not stated. The abstract does not say how the 31.0% to 68.5% range maps to individual tasks or to which hardware platform. No numeric value for 'near-zero forgetting' is given, and the number of tasks in the sequences and the number of retained episodes are not stated.

Risks and caveats

The training-time reduction is stated for real-robot training only; no figures are given for simulation. No specific robot foundation model, VLA, robot hardware or simulator is named, so it is unclear how broadly the method transfers. Because the baseline and the per-task breakdown are missing, the wide 31.0% to 68.5% range cannot be read as a typical gain. The claims have not been independently checked in the material available.