NVIDIA has released four open-weight Cosmos 3 checkpoints for generating interactive simulated worlds, including synchronised camera and LiDAR scenes for driving and video predictions for robot actions. The practical prize is more controllable training and testing environments for AI systems that have to act in the physical world, not just describe it.
NVIDIA Watch analysis
What happened
The models, announced in a post on Hugging Face’s blog, are Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer and Cosmos3-Nano-Sim-Bimanual. NVIDIA says each has 16 billion trainable parameters and uses the OpenMDW1.1 licence, which permits commercial and non-commercial use under its terms.
Transfer-Auto can generate driving scenes across as many as 11 camera views, with joint RGB and LiDAR generation. Sim-Auto is its streaming counterpart, generating successive camera and LiDAR observations as driving controls change. Sim-Transfer turns simulator controls and text prompts into generated visual observations, while Sim-Bimanual predicts how a scene changes in response to two-arm robot actions.
Why it matters
These are tools for building and testing physical-AI systems in simulation. Developers can vary weather, lighting, road layouts and traffic behaviour for driving scenarios, or explore how a robot’s actions alter its surroundings. That offers a way to create more varied training or evaluation data without treating every real-world condition as a test opportunity.
There are important differences between the models: some generate offline data, while others are designed for streaming interaction. For Sim-Bimanual, NVIDIA lists support for YAM and AgiBot robot embodiments. The published setup was tested on one GB200, which is a description of that test environment, not a stated minimum hardware requirement.
Our read
This is a substantive set of releases, not four names stapled to one demo. The open checkpoints and post-training recipes give developers a starting point for adapting the models to their own sensors, driving domains or robot embodiments. But a convincing simulation is not automatically a faithful one: NVIDIA says the work ahead includes measuring where the models support policy learning and evaluation, and where they fall short.
What to watch
- Whether independent evaluations show that simulated camera and LiDAR outputs remain useful across changing scenes.
- How well policies trained or tested in these environments transfer to real vehicles and robots.
- What compute and data are needed to adapt the checkpoints to different sensors and robot embodiments.
Discussion spark: For physical-AI testing, should teams prioritise richer simulations, or spend more effort proving that simulated results transfer reliably to the real world?
Sources and evidence
- New NVIDIA Cosmos 3 Open Models for Interactive World Simulation (5 November 2026, 19:35 UTC)
Independent WittyWires tracker for public updates about NVIDIA. Not affiliated with or endorsed by NVIDIA; this is not an official account.