NVIDIA — One Checkpoint, Three Jobs for Robots
Episode notes
AWS and NVIDIA's “physical AI model factory” trains robot and self-driving models in a continuous loop on one cluster. One Cosmos 3 checkpoint can imagine the future, label actions in raw video, and drive the robot.
Key takeaways:
- One cluster, closed loop. Synthetic-data generation, post-training and evaluation share one SageMaker HyperPod cluster and storage instead of separate clusters that sit idle.
- One token stream. Cosmos 3 treats video, images, actions and sound as a single sequence, with a reasoner and a generator joined by attention at every layer.
- Three jobs, one checkpoint. Which tokens start as noise decides the mode: world model (predicts the next frames), action labeler (infers the action behind a change), or policy (controls the robot).
- Sizes. Super is 64B on a 32B Qwen3-VL backbone, adapted with rank-16 LoRA. Nano is 16B, and Edge is a 4B model for on-device use.
- Efficiency. Super stayed near-linear (about 0.97 of linear or better) from one to four 8×H200 nodes. MFU was about 0.50 for Super and 0.24 for Nano.
Chapters:
00:00 Robots that stop learning
00:45 AWS and NVIDIA's model factory
01:32 The closed loop
03:04 Why separate clusters waste GPUs
04:35 Cosmos 3: one token stream
06:08 Reasoner and generator
08:26 Model tiers and LoRA
09:59 Three modes, one checkpoint
12:16 Running on SageMaker HyperPod
14:33 Scaling and goodput
16:03 Checkpoint timing (Young/Daly)
16:50 What's new
17:36 Industry comparisons
19:53 Recap
Sources:
- Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod (AWS Machine Learning Blog, 2026)
Related episode: E4: Google — Ending the Six-Hour Forecast Gap
Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.
Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.
Made with AI-generated voices; curated by Peter Liu from the sources above.