Glonce

CaPtain trains a 4B Qwen3.5 model to control robots by writing code

researchThe Glonce editorsHugging Face🔥55Glonce rating55Impact40Novelty60Relevance90Surprise30Players15Credibility60
CaPtain trains a 4B Qwen3.5 model to control robots by writing code

A paper on Hugging Face presents CaPtain, an open post-training pipeline that turns a small open vision-language model into a code-as-policy (CaP) robot agent. In code-as-policy, the model controls a robot by writing code against robot APIs and reading back the results. The authors start from an observation about other agent domains: coding, web and tool-use agents are now routinely trained with the same recipe, supervised fine-tuning (SFT) to learn the interface and then multi-turn reinforcement learning (RL) to master it. They argue the same recipe should apply to robot manipulation, yet existing CaP agents sidestep training. They either prompt closed frontier models or hand manipulation to pre-trained vision-language-action models (VLAs) and high-level skills, while small open VLMs fail outright.

CaPtain trains and deploys the policy inside CaPtain-Harness, an environment that exposes low-level perception, geometry and guarded-motion primitives across both simulation and real hardware. At test time the system needs no VLA, no frontier model and no privileged state. Training has two stages. First, the base model is aligned with SFT on synthetic trajectories generated with GPT-6 Astra. Second, multi-turn RL is applied.

The RL stage has a credit-assignment problem: the agent makes tens of requests per episode, their physical effects often show up turns later, and the only signal is a single end-of-episode outcome. The authors' answer is a physical critic. It reads privileged simulator state that the policy never sees, and it credits each request by how much that request changes the predicted chance of success.

The results start from Qwen3.5-4B, which scores 0% zero-shot. The resulting CaPtain-4B reaches 55.4% held-out LIBERO success after SFT and 73.8% after RL. On LIBERO-PRO it reaches 78.5%, which the authors say surpasses several frontier-model agents, and the RL gains persist across three perturbation suites, ranging from +17.6 to +21.2 points. Without training on the target domain, it attains 83.3% on SimplerEnv WidowX, exceeds frontier models on CaP-Bench, and runs on a real robot after simulation-only adaptation. The authors release all code, data and model weights.

Key facts

  • CaPtain is an open post-training pipeline: SFT on synthetic trajectories generated with GPT-6 Astra, then multi-turn RL, turning a small open VLM into a code-as-policy robot agent.
  • Starting from Qwen3.5-4B at 0% zero-shot, CaPtain-4B reaches 55.4% held-out LIBERO success after SFT and 73.8% after RL.
  • A physical critic reads privileged simulator state the policy never sees and credits each agent request by how much it changes the predicted chance of success.
  • On LIBERO-PRO it reaches 78.5%, which the authors say surpasses several frontier-model agents; RL gains across three perturbation suites range from +17.6 to +21.2 points.
  • It attains 83.3% on SimplerEnv WidowX without target-domain training, runs on a real robot after simulation-only adaptation, and all code, data and weights are released.

Why it matters

Agents for coding, web and tool use are trained with SFT followed by multi-turn RL, but the authors say code-as-policy robot agents have skipped that step. Existing ones prompt closed frontier models or lean on pre-trained VLAs and high-level skills, and small open VLMs fail outright. CaPtain's claim is that the standard recipe works for robots too: a 4B model that scores 0% zero-shot reaches 73.8% held-out LIBERO success after RL, with no VLA, frontier model or privileged state at test time. The new piece is the physical critic, which addresses the credit-assignment problem of crediting tens of requests from one end-of-episode outcome.

Who it affects

Robotics researchers working on code-as-policy and manipulation benchmarks such as LIBERO, LIBERO-PRO, SimplerEnv WidowX and CaP-Bench are the direct audience. So are teams that want robot agents built on small open models rather than closed frontier models or pre-trained VLAs. The authors state that the code, data and weights are released.

How to use it

The authors say they release all code, data and model weights, so the pipeline and the CaPtain-4B model can in principle be reproduced or built on. The abstract gives no repository or download link. The setup is: a small open VLM as base (here Qwen3.5-4B), the CaPtain-Harness environment for simulation and real hardware, SFT on synthetic trajectories, then multi-turn RL with the physical critic.

How solid is it

This is a preprint abstract on Hugging Face, and all figures are the authors' own reports. The numbers are specific: 0% zero-shot, 55.4% after SFT, 73.8% after RL on held-out LIBERO, 78.5% on LIBERO-PRO, and 83.3% on SimplerEnv WidowX. The comparisons with frontier models are stated only in general terms. The abstract does not name which frontier models were surpassed or exceeded, nor give their scores, and it gives no CaP-Bench score and no success rate for the real-robot run.

Risks and caveats

The abstract does not say which three perturbation suites were used or which gain belongs to which suite, only the +17.6 to +21.2 point range. It gives no training cost, compute, dataset size or number of trajectories. The SFT data come from GPT-6 Astra, so the pipeline's first stage depends on that model, even though the deployed agent needs no frontier model at test time. The physical critic uses privileged simulator state during training, and the real-robot result comes after simulation-only adaptation, with no success rate given.

“We release all code, data, and model weights.”

— CaPtain paper abstract