Nvidia GR00T 2 robot model learns new tasks from human video

Teaching a robot a new task usually means engineers spend hours controlling the machine remotely. This teleoperation is expensive and slow, and it needs its own data for every robot model. In a new research report on GR00T 2, Nvidia introduces a foundation model designed to shorten that process by transferring human movements from video to robotic arms.

Unlike earlier approaches, including its predecessor GR00T 1, GR00T 2 does not translate camera images and language commands directly into motor movements. Nvidia says the software is what it calls a world-action model: it calculates the next movement and a short video of the immediate future at the same time. For example, it predicts how the camera image should change when a robotic arm picks up an object. The Decoder writes that this video prediction should let the model transfer movement sequences between different robot form factors. Nvidia trained the system on a cluster with more than a thousand GB300 chips, using about 400,000 hours of video of people performing various tasks while wearing head-mounted cameras.

To teach a robot a new task, the researchers say it should be enough for a person to demonstrate the steps with data gloves, specialized hand grippers or a finger exoskeleton. During training, these recordings are combined with general motion data from the target robot. The AI then generates the robotic arm instead of the human arm in its video prediction and derives the matching control commands from that.

In lab tests, a robot with five-fingered hands lit a match, sliced a banana and tossed packages onto a conveyor belt this way, without having been teleoperated for those specific tasks. Success rates varied between 10 and 80 percent depending on the task. From a single video of a human hand movement, the system also learned to hammer nails by way of a computer simulation. For 50 selected tasks where the model already performed well after just a few examples, less than three minutes of conventional training raised the success rate from 40 to 95 percent.

For longer assembly work, Nvidia pairs GR00T 2 with a higher-level language model, with OpenAI's GPT-6 Astra given as an example of such a model. GR00T 2 handles the gripping movements, while the language model breaks the task into steps, color-codes grip points in the camera image and launches helper routines when a part falls off. When researchers placed loose toy-model parts on a table in no particular order, the combined system assembled them in at least 97 percent of trials. Without the corrective language model and additional fine-tuning, the toy car assembly succeeded in only 23 out of 100 attempts. The team also trained precision tasks such as plugging in network cables or chargers through repeated trial and error on the real device.

For bipedal robots, Nvidia built a slimmed-down variant, GR00T 2 Turbo, which processes fewer image details and runs directly on the Jetson Thor onboard computer of a humanoid robot. In testing it scooped popcorn, cleared tables and wiped up liquids with a towel under its foot. That last task succeeded in just under half of all attempts.

The research team names significant hurdles before the system is ready for reliable real-world use. The model does not yet process direct sensor data for touch and sometimes misses subtle errors such as half-turned screws. The higher-level language model also slows things down considerably: during trash sorting, the robot stood idle for roughly a third of the total time, waiting an average of more than seven seconds for the planning model's next computation.

Key facts

GR00T 2 is what Nvidia calls a world-action model: it computes the next movement and a short video of the immediate future together, unlike GR00T 1, which maps images and language commands straight to motor movements.
Training used a cluster with more than a thousand GB300 chips and about 400,000 hours of head-mounted-camera video of people doing tasks; new tasks can be shown with data gloves, hand grippers or a finger exoskeleton.
Lab success rates ranged from 10 to 80 percent by task; for 50 selected tasks, under three minutes of conventional training lifted success from 40 to 95 percent.
Paired with a higher-level language model, the system assembled loose toy parts in at least 97 percent of trials, against 23 out of 100 for toy car assembly without the corrective model and extra fine-tuning.
Limits named by the team: no touch-sensor input yet, missed subtle errors such as half-turned screws, and a planning model that left the robot idle for roughly a third of the time in trash sorting.

Why it matters

Teleoperation is the costly step in teaching robots: it is slow, expensive and needs separate data for each robot model. GR00T 2 is Nvidia's attempt to replace part of it with human demonstrations. The design change is the world-action approach, where the model predicts a short future video alongside the next movement, rather than mapping images and language straight to motor commands as GR00T 1 and earlier approaches did. The video prediction is meant to let movements carry over between different robot form factors, though that is stated as an expectation, not a demonstrated result.

Who it affects

Mainly teams that build and train robots and now rely on teleoperation for training data. Under Nvidia's approach, a person wearing data gloves, using specialized hand grippers or a finger exoskeleton could show the steps instead. The slimmed-down GR00T 2 Turbo, which runs on the Jetson Thor onboard computer, points at makers of humanoid robots.

How to use it

The article does not say whether GR00T 2 or its weights are released, open-sourced or commercially available, nor any price. What it describes is the workflow: a person demonstrates the steps with data gloves, grippers or an exoskeleton; those recordings are combined in training with general motion data from the target robot; the model then generates the robot arm in place of the human arm in its video prediction and derives the control commands. For longer assembly jobs, Nvidia adds a higher-level language model to split the task into steps, mark grip points and trigger helper routines. Precision tasks such as plugging in cables or chargers were trained by repeated trial and error on the real device.

How solid is it

These are Nvidia's own lab results from a research report, relayed by The Decoder. Success rates varied widely, from 10 to 80 percent by task, and the article does not say which task sat at either end. The 40 to 95 percent jump applies only to 50 selected tasks where the model already did well after a few examples. The 97 percent assembly figure is for the combined system with the language model; the 23 out of 100 figure is for toy car assembly without the corrective model and additional fine-tuning. The article gives no numerical comparison with GR00T 1 or with competitors.

Risks and caveats

The research team itself lists hurdles before reliable real-world use. The model does not yet take direct touch-sensor data and sometimes misses subtle errors such as half-turned screws. The language model that helps with errors also slows the robot: in trash sorting it stood idle for roughly a third of the time, waiting more than seven seconds on average for the planning model. The humanoid's towel-wiping task succeeded in just under half of attempts. The article does not say whether GPT-6 Astra was the model actually used in the experiments; it is given only as an example.