RELEASE · MODELS · #2291
NVIDIA releases Cosmos3-H-Dreams — real-time generative surgical simulator trained on cholecystectomies
NVIDIA announced Cosmos3-H-Dreams, a Cosmos 3–based generative simulator trained on synchronized clinical cholecystectomy video and robot kinematics from the CMR Surgical Versius platform that supports live kinematic control and runs interactively on a single RTX PRO 6000 GPU. The release uses a teacher-to-student distillation pipeline, two student checkpoints (four-step and two-step denoising schedules), and FlashDreams accelerated streaming inference to trade off visual quality, temporal consistency, and low-latency control.
KEY POINTS
- NVIDIA announced Cosmos3-H-Dreams, a Cosmos 3–based generative simulator trained on synchronized clinical cholecystectomy video and robot kinematics from the CMR Surgical Versius platform that supports live kinematic control and runs interactively on a single RTX PRO 6000 GPU.
- The release uses a teacher-to-student distillation pipeline, two student checkpoints (four-step and two-step denoising schedules), and FlashDreams accelerated streaming inference to trade off visual quality, temporal consistency, and low-latency control.
- If validated, this demonstrates that action-conditioned generative video models can produce low-latency, interactive simulations of complex intraoperative scenes, offering a potential alternative or complement to numerical surgical simulators while requiring careful biomechanical validation.
WHY IT MATTERS
If validated, this demonstrates that action-conditioned generative video models can produce low-latency, interactive simulations of complex intraoperative scenes, offering a potential alternative or complement to numerical surgical simulators while requiring careful biomechanical validation.