Ctrl-World: A Controllable Generative World Model for Robot Manipulation

Understand

Generalist robot policies can now perform a wide range of manipulation skills, but evaluating and improving their ability with unfamiliar objects and instructions remains a significant challenge.

• Rigorous evaluation requires a large number of real-world rollouts, while systematic improvement demands additional corrective data with expert labels.
• Both of these processes are slow, costly, and difficult to scale.
• World models offer a promising, scalable alternative by enabling policies to rollout within imagination space.