Table of Contents
NVIDIA has introduced GR00T 2, an open world action model built on Cosmos that learns robot manipulation by predicting both physical actions and their visual consequences. The model uses human video, handheld-gripper demonstrations, and robot data to transfer skills to machines that have never demonstrated those tasks themselves.
Unveiled at the Conference on Robot Learning in Austin, GR00T 2 reached 95% average success across 50 tasks after fine-tuning on five demonstrations per task, NVIDIA reported. Without task-specific demonstrations from the target robot, it achieved 40% average success and completed 40 of the 50 tasks at least once.
NVIDIA's CoRL robotics announcements also cover Isaac Lab 3.0, the Isaac Lab-Arena 0.4 developer preview, and two open industrial benchmarks. Together, they address different parts of robot development: learning physical skills, refining them in simulation, and measuring how reliably they work under changing conditions.
How GR00T 2 Learns From Human Activity
A world action model learns how a scene changes alongside the movements that produce that change. Rather than only mapping a camera image and instruction to a robot command, GR00T 2 jointly predicts future video and actions.
Its diffusion transformer takes recent video, a language instruction, and the robot's state, then generates a future visual sequence and a chunk of actions together. The robot executes the action chunk and repeats the process with fresh observations. Small, robot-specific output heads translate the shared model's predictions into commands for different bodies.
That architecture gives human video a direct role in learning physical interactions, even when the footage contains no robot motor commands. NVIDIA's GR00T 2 research details list a pretraining mixture of 402,730 hours of human egocentric video, 10,184 hours of handheld-gripper demonstrations across 2,897 tasks, and 23,500 hours of robot teleoperation.
The 100,000 hours of open human video highlighted in the announcement are a publicly available portion of the larger human-video dataset, not the entire training mixture. That subset is available as Human-1M-100K on Hugging Face.
In one human-to-robot transfer experiment, a robot with two five-fingered hands learned to line a laboratory funnel with filter paper and wet it with a wash bottle, succeeding in 65% of trials. The task demonstrations came from humans, while task-agnostic robot data taught the system how its own body moved. In another experiment, demonstrations with a handheld gripper transferred to a YAM arm, which completed toy dump-truck assembly in 60% of trials.
Few-Shot Adaptation and Agent-Controlled Work
The 50-task evaluation tested skills present in human video or other robots' data but absent from the target robot's task demonstrations. NVIDIA reported the following results:
| Target-Robot Demonstrations per Task | Average Success Across 50 Tasks |
|---|---|
| None | 40% |
| One | 71% |
| Five | 95% |
With five demonstrations, each task used less than three minutes of robot data and a 30-minute LoRA fine-tune on a single RTX 5090. LoRA adapts a model by training a small set of additional parameters rather than updating all its weights. In that setting, 37 of the 50 tasks succeeded in every trial.
A separate reasoning agent can call GR00T 2 through an API, check progress, and invoke recovery skills when execution stalls. The agent handles task sequencing and verification, while GR00T 2 performs the physical manipulation.
On a YAM arm, the combined system completed 98 of 100 toy-plane assemblies, 97 of 100 toy-tractor assemblies, and 97 of 100 Hot Wheels car assemblies. Each trial began with parts dumped into a cluttered layout. The same post-trained policy without the agent succeeded in 71–84 of 100 trials across those tasks.
NVIDIA also detailed GR00T 2 Turbo, a smaller variant with a 2-billion-parameter transformer and a more compressed video representation. It runs onboard a Unitree G1 humanoid using Jetson Thor, with median prediction latency of 124 milliseconds. Turbo plans chunks of whole-body motion, while the SONIC controller converts those plans into joint targets at 50 Hz.
Industrial evaluations include Nexuni's laundry handling, Foxconn's server-manufacturing automation, and Siemens' industrial use cases. MicroAGI is using reinforcement-learning post-training to teach GR00T 2 Ethernet cable swapping on a YAM arm, with agents building and calibrating the simulation environments in Isaac Lab 3.0 and Newton.
Isaac Lab 3.0 Expands Simulation and Post-Training
Isaac Lab 3.0 provides a common task interface across multiple physics, rendering, and visualization backends. Developers can switch supported tasks between PhysX and Newton without rewriting their task logic, and many workflows can run without installing the full Isaac Sim application.
The framework adds agent-assisted environment setup and a reinforcement post-training library for foundation models such as GR00T. Reinforcement learning improves a policy through repeated attempts and success signals, complementing learning from demonstrations.
Newton integration supports deformable objects, including cables and cloth, as well as articulated systems such as robotic hands. In manipulation, bending, contact, and finger coordination determine whether actions such as inserting a connector work.
On a vision-based dexterous-manipulation benchmark, NVIDIA reported up to 2.5 times the training throughput for Isaac Lab 3.0 with Newton compared with Isaac Lab 2.3 with PhysX.
AlphaNIST, built in Isaac Lab, illustrates the training approach on precision assembly. A single policy learned all 20 NIST Assembly Task Board tasks using a yes-or-no success signal, reaching more than 98% training success versus 81% for separate per-task policies, with roughly 39 times less GPU time.
Its Success-Guided Sampling method, developed by the University of Washington and NVIDIA, concentrates practice on tasks the policy succeeds at about half the time. This directs training toward skills it is still learning rather than repeatedly practicing tasks it has already mastered.
New Benchmarks Test Industrial Dexterity
Isaac Lab-Arena 0.4, arriving as a developer preview at CoRL, lets teams construct repeatable evaluations and run policies across thousands of variations in parallel. Its failure analyses identify conditions that should inform the next training run.
DexBench, developed by RLWRLD with NVIDIA, includes 15 tasks across retail, logistics, manufacturing, and hospitality scenes. It tests how performance changes with different objects, spatial arrangements, and physical properties.
VABAR, developed by UC Berkeley and NVIDIA, covers six industrial tasks with standardized variations, difficulty levels, and success metrics. The benchmarks give developers structured ways to test adaptability beyond a single successful demonstration.
Another Berkeley project, Graph-as-Policy, uses agents to connect individual skills into interpretable robot programs. Agents test those programs in simulation and revise them based on failures, giving engineers code they can inspect before deployment.
Ambi Robotics, Bosch Research, and Siemens are exploring Graph-as-Policy for industrial workflows. The work complements GR00T 2's agent interface, where one system develops and tests the task program while learned manipulation skills supply the physical actions.
NVIDIA plans to make GR00T 2 code available in December. At CoRL in Austin, its researchers are demonstrating GR00T, Isaac Lab, and Newton at booth 404 through November 11. Jim Fan's November 9 keynote, scheduled for 1:50 p.m. Central, will cover world action models and agentic robotics.
