This page is an AI translation of the Japanese original. If anything differs, the Japanese version prevails.
NVIDIA Releases Framework for Applying Cosmos to Interactive World Simulation
NVIDIA has published on arXiv a common framework that uses its Cosmos world foundation model for real-time interactive simulation. Code and model checkpoints have also been released.
This article was generated by AI and published automatically without prior human review.
- Announced
- 2026.11.04 (local time)
- Use cases
- Robotics, Research, Autonomous driving
- Sources
- 1 (en) (1 primary)
Key points
- A framework that generates video sequentially in response to actions and scene controls, letting users and agents interact with a virtual environment
- Supports both predicting the visual consequences of actions and generating observations from the physical state of an external simulator
- Code and checkpoints are available on GitHub and Hugging Face
Contents
What happened
NVIDIA has published a common framework for adapting its Cosmos World Foundation Model to real-time interactive world simulation for physical AI, in a paper titled "Interactive World Simulation with Cosmos for Physical AI." The paper was posted to arXiv on November 4, 2026.
How it works
According to the paper's abstract, the framework continuously generates visual observations in response to actions and scene controls. This allows humans and embodied agents to interact with a virtual environment that keeps changing.
Why it matters
This research points toward building video-generation-based virtual environments on a common foundation for training and evaluating robots and autonomous driving. It may be usable for validation before real-machine testing, but accuracy and suitability for practical use cannot be confirmed from the available material.
Examples
- Camera navigation and robot manipulation
- Controllable video transfer
- Multi-view autonomous vehicle (AV) simulation
How to try it
- From the paper's arXiv page, go to the official project site and GitHub repository and check the published samples
- Check the checkpoint types and licenses in the Hugging Face collection
Caveats and open questions
- The material is limited to the paper's abstract, so figures such as model size, latency, frame rate and evaluation results cannot be confirmed.
- Required computing resources and license terms cannot be confirmed from the material.
- The material does not say whether the framework works with real-world 3D scans or 3DGS.
- The post on X served as a lead on the topic; the article's facts are based on the arXiv abstract.
Glossary
- World Foundation Model (WFM)
- A large-scale model that predicts and generates changes in an environment, for example as video. It is used as a foundation for training and evaluating physical AI.
- Physical AI
- A general term for AI that operates in the real world, such as robots and autonomous vehicles.
- Video transfer
- A process that converts an input video into another video by changing its appearance or conditions while preserving its structure.
Sources
Primary sources (publisher, paper, repository)
- Interactive World Simulation with Cosmos for Physical AIarXiv（arxiv.org）・en
The X post that surfaced this story
This article was generated in Japanese by AI from the sources listed (primarily primary sources) and published automatically, without prior human review, after automated checks such as source matching (how we use AI). Figures and specifications are as stated by the publishers and have not been independently verified by us. Please consult the original sources. To report an error, use our contact form; we will review it promptly and correct it. This page is an AI translation of the Japanese original. If anything differs, the Japanese version prevails. Japanese original
From the operator点群×BIM×3D/4D Gaussian（IZUTSUYA Inc.）