Show-Harness lets frontier VLMs control robots zero-shot

Foundation vision-language models (VLMs) already reason broadly about the world, but turning that reasoning into robot control has been difficult. A new paper, Show-Harness, by Yanzhe Chen and co-authors, proposes an Embodied Harness: a compact semantic interface that links a VLM's intent directly to robot action, letting the model 'play' a robot instead of requiring a purpose-built policy for each new task or platform.

The harness works by exposing discrete semantic action units, a vocabulary of actions a VLM can reason over the same way it reasons over language, rather than raw low-level robot commands. Embodiment-specific interpreters then deterministically translate those semantic actions into the local commands a given robot actually executes. The VLM itself stays directly responsible for the fine-grained physical decisions; the interpreter only handles the mechanical translation into that robot's own action space.

Through this interface, the authors demonstrate two distinct deployment paths. Closed-source frontier VLMs can be unlocked for zero-shot robot control directly, with no additional training at all. Small-scale open-source VLMs can instead be adapted for low-cost deployment with just a few GPU-hours of fine-tuning, rather than the larger, embodiment-specific training runs such adaptation usually requires; the paper does not give an exact GPU-hour figure.

The authors also introduce GUMI, a GUI Manipulation Interface that extends the same semantic action space to demonstration collection. Instead of physical teleoperation hardware, humans and agents can generate robot demonstrations by interacting with a GUI, across different robot embodiments, using the same semantic actions the harness already exposes.

In what the authors describe as extensive experiments, VLM agents equipped with Show-Harness generalized robustly across tasks, embodiments, and environments, and outperformed representative agentic and vision-language-action (VLA) approaches. The paper does not name the specific robot platforms, tasks, or benchmarks used, nor does it give success rates or margins for these comparisons. The authors' conclusion is that the right interface, not extra model capacity or costly embodiment-specific pretraining, is what unlocks embodied capability already latent in foundation VLMs.

Key facts

Show-Harness is an Embodied Harness, by Yanzhe Chen and co-authors, that lets vision-language models (VLMs) control robots through a compact semantic interface linking intent to action.
The harness exposes discrete semantic action units that VLMs reason over directly, while embodiment-specific interpreters deterministically ground them into local robot actions, keeping the VLM responsible for fine-grained physical decisions.
Closed-source frontier VLMs get zero-shot robot control with no additional training; small-scale open-source VLMs can be adapted for low-cost deployment with just a few GPU-hours of fine-tuning.
A companion tool, GUMI (GUI Manipulation Interface), extends the same semantic action space to GUI-based demonstration collection, letting humans and agents generate robot demonstrations across embodiments without specialized teleoperation hardware.
In the authors' experiments, Show-Harness-equipped agents generalized across tasks, embodiments, and environments and outperformed representative agentic and VLA approaches, though no specific numbers, platforms, or benchmarks are given.

Why it matters

Foundation VLMs already reason broadly about the world; the open problem has been turning that reasoning into robot control without either retraining the model or building a bespoke policy for each new task and platform. Show-Harness reframes this as an interface problem rather than a model-capacity problem: a compact set of semantic action units a VLM can already reason over, paired with deterministic, embodiment-specific interpreters that translate those actions into a given robot's own commands. The VLM keeps direct responsibility for the fine-grained physical decisions; the interpreter only handles the mechanical translation. The authors argue this shows that the right interface, not additional model capacity or costly embodiment-specific pretraining, is what is needed to unlock embodied capability already latent in foundation VLMs.

Who it affects

Researchers and engineers building embodied agents or robot foundation models are the direct audience. Show-Harness lets them plug an existing closed-source frontier VLM into a robot with zero-shot control and no additional training, or adapt a small open-source VLM for low-cost deployment with just a few GPU-hours of fine-tuning, which matters for teams without access to large training budgets. GUMI extends the reach further, to anyone who collects robot demonstration data: because it moves demonstration collection onto a GUI built on the same semantic action space, both human operators and agents can generate demonstrations across different robot embodiments without specialized teleoperation hardware.

How to use it

The paper describes two deployment paths rather than a single product. Point a closed-source frontier VLM at the harness and it works zero-shot: the model reasons directly over Show-Harness's semantic action units, and the embodiment-specific interpreter grounds its choices into that robot's own commands, with no training step at all. For a small-scale open-source VLM, the authors report adapting it for low-cost deployment with just a few GPU-hours of fine-tuning, an exact figure the paper does not give. Separately, GUMI opens the same semantic action space to demonstration collection through a GUI, letting people or agents generate 'play' demonstrations across embodiments without needing physical teleoperation hardware.

How solid is it

The abstract's evidence is qualitative: 'extensive experiments' are said to show Show-Harness agents generalizing robustly and outperforming representative agentic and VLA baselines, but no success rates, task counts, or performance margins are given, and no specific robot platforms, benchmarks, or environments are named beyond the generic 'tasks, embodiments, and environments.' No date or publication venue appears in the text either, so the claims currently rest on the authors' own description, with any supporting figures presumably in the full paper rather than the abstract.

Risks and caveats

Treat the performance claims as the authors' own summary rather than a benchmarked result: 'outperforming representative agentic and VLA paradigms' and 'a few GPU-hours' of fine-tuning both appear in the abstract without attached numbers. By design, the VLM keeps direct responsibility for the fine-grained physical decisions, while the interpreter only grounds the model's chosen actions deterministically into a given robot's local commands.

“These results suggest that the right interface can unlock substantial embodied capability from foundation VLMs, without requiring additional model capacity or costly embodiment-specific pretraining.”

— the authors of the Show-Harness paper