Cosmos-H-Surgical: Learning Surgical Robot Policies from Videos via World Modeling

Understand

Data scarcity remains a fundamental barrier to achieving fully autonomous surgical robots.

• While large scale vision language action (VLA) models have shown impressive generalization in household and industrial manipulation by leveraging paired video action data from diverse domains, surgical robotics suffers from the paucity of datasets that include both visual observations and accurate robot kinematics.
• In contrast, vast corpora of surgical videos exist, but they lack corresponding action labels, preventing direct application of imitation learning or VLA training.
• In this work, we aim to alleviate this problem by learning policy models from Cosmos-H-Surgical, a world model designed for surgical physical AI.