Glonce

PARTS retrains pretrained robot policies only on the subtasks they keep failing

researchThe Glonce editorsHugging Face🔥71Glonce rating71Impact65Novelty70Relevance95Surprise50Players30Credibility85
PARTS retrains pretrained robot policies only on the subtasks they keep failing

Robots trained with a pretrained foundation policy often get most of a long, multi-step task right but fail repeatedly at a handful of specific steps. The usual fix, collecting more full-task demonstrations for supervised fine-tuning, wastes effort because it makes operators repeat behaviors the robot already does well. Reinforcement learning (RL) fine-tuning is a more targeted alternative, but the authors say existing RL approaches struggle to solve long-horizon tasks when they only get a sparse reward at the very end.

The paper introduces PARTS (Policy Adaptation with RL on Targeted Subtasks), a real-world RL framework built to concentrate training effort exactly at those failure points. The original pretrained policy stays frozen and keeps supplying its normal actions throughout the task. On top of it, agent-generated selectors detect when the robot has entered a bottleneck subtask, and success verifiers check whether that subtask was completed; a residual correction policy is activated only there and receives a local reward for that outcome. Because the reward is tied to the subtask rather than the whole task, the system can learn from successful subtask attempts even when full-task successes are rare. Training alternates online RL rollouts with retraining that is reweighted toward successful attempts, and each newly retrained residual policy is redeployed to collect more data. Humans are still involved, but only to identify the bottleneck subtasks during setup and to perform physical resets between rollouts.

The authors tested PARTS on bimanual YAM robot tasks and on single-arm Franka robot tasks. Complete-task success rose from 32% to 61% on the bimanual YAM tasks and from 50% to 95% on the single-arm Franka tasks, using an average of tens of minutes of real-world RL rollouts per task. Against other existing real-world RL fine-tuning methods run under the same robot-rollout budget, PARTS raised full-task success by more than 25% while needing less human involvement.

Key facts

  • PARTS fine-tunes a frozen pretrained robot policy only at the specific subtasks it keeps failing, using agent-generated selectors and success verifiers instead of retraining the whole task.
  • On bimanual YAM tasks, complete-task success rose from 32% to 61%.
  • On single-arm Franka tasks, complete-task success rose from 50% to 95%.
  • Training used an average of tens of minutes of real-world RL rollouts per task.
  • Under the same robot-rollout budget, PARTS beat existing real-world RL fine-tuning methods by more than 25% in full-task success while requiring less human involvement.

Why it matters

Foundation policies for robots already handle most of a long task well; what breaks them is a small number of stubborn subtasks. Collecting more full demonstrations to fix those few steps forces operators to redo everything the robot already gets right, which is expensive and slow. PARTS targets training precisely where the policy fails instead of retraining the whole behavior, which is the more efficient way to close that last gap.

Who it affects

The immediate audience is robotics researchers and engineers building on pretrained manipulation policies who need to close a persistent failure gap without collecting large new demonstration sets. It also matters to teams evaluating real-world RL fine-tuning methods, since PARTS is compared directly against existing approaches under a matched rollout budget.

How to use it

PARTS keeps the pretrained policy frozen and adds a residual correction layer that only activates at identified bottleneck subtasks. A human's role during training is limited to flagging which subtasks are the bottlenecks at setup and performing physical resets between rollouts; the selectors, verifiers and reward assignment run automatically from there, with each retrained residual policy redeployed to keep collecting experience.

How solid is it

The results come from real-world robot rollouts, not simulation, on two distinct platforms: bimanual YAM and single-arm Franka. The reported gains are large and consistent across both platforms, and the paper also reports a direct, same-budget comparison against existing real-world RL fine-tuning methods, where PARTS still comes out more than 25% ahead. The source text does not state author names, institutional affiliations, publication venue or an exact rollout-time figure beyond 'tens of minutes' on average.

Risks and caveats

The method still depends on humans to identify which subtasks are the bottlenecks and to perform physical resets, so it is not fully autonomous. The reported figures come from two specific robot platforms and task sets; how the approach generalizes to other robots, longer task chains or subtler failure modes beyond these tested cases is not addressed in the source.