NVIDIA Cosmos

NVIDIA Cosmos is a family of open world foundation models built for physical AI — robots, autonomous machines and vision systems that must perceive, reason about and act in the real world. A Cosmos model can be used as a vision-language model, as a simulator that predicts how a scene evolves frame by frame, and as the backbone of world action models that generate robot behaviours.^[1][2]

History

NVIDIA introduced Cosmos at the start of 2025 as world foundation models trained on billions of frames of video and sensor data, aimed at the data bottleneck that limits robotics: real-world footage of rare events — a robot dropping an object, a near-miss on a highway — is scarce, expensive and dangerous to collect.^[2]

On 31 May 2026, at GTC Taipei, NVIDIA released Cosmos 3, described as an open frontier omnimodel that natively understands and generates text, images, video, ambient sound and actions. It is built on a mixture-of-transformers architecture that pairs a reasoning transformer with a generation transformer, so the model reasons about object interactions, motion and spatial-temporal relationships before producing video or action trajectories. The same release founded the NVIDIA Cosmos Coalition, bringing together world-model builders including Agile Robots, Black Forest Labs, Generalist, LTX, Runway and Skild AI.^[1]

Key Concepts and Technology

Three capabilities are bundled into one system. As a vision-language model it answers questions and reasons over visual input. As a world model it simulates a physical environment and rolls the state forward, which lets developers evaluate policies without deploying them. As a source of world action models it supplies the trajectories that teach a robot a task.^[1]

Training used one of the largest multimodal physical AI datasets assembled by the company — billions of samples across text, image, video, sound and action — spanning robotics, physics, human motion, driving, warehouse safety and spatial reasoning, released alongside agent skills for neural scene reconstruction and defect-image generation.^[1] The practical claim is that Cosmos cuts physical-AI training and evaluation cycles from months to days by generating synthetic data for scenarios that are hard to capture, and by serving as a physics-plausible test bed before hardware trials.

Applications and Impact

The main users are autonomous-vehicle developers, who can pre-train and stress-test policies on generated traffic scenarios; robot makers, who synthesise demonstrations of pick-and-place, sorting and manipulation; and industrial vision teams, who augment inspection datasets with rare defect imagery. Analysts rank Cosmos among the leading world foundation models of 2026, alongside NVIDIA's own GR00T humanoid line and Google DeepMind's Genie family, with world models increasingly positioned as the simulation layer of the physical-AI stack.^[3]

>See Also

• World Models
• Physical AI
• NVIDIA
• Synthetic Data
• Digital Twin
• NVIDIA Blackwell
• AI in Malaysian Manufacturing