Evaluating Gemini Robotics Policies in a Veo World Simulator

Understand

Generative world models hold significant potential for simulating interactions with visuomotor policies in varied environments.

• Frontier video models can enable generation of realistic observations and environment interactions in a scalable and general manner.
• However, the use of video models in robotics has been limited primarily to in-distribution evaluations, i.e., scenarios that are similar to ones used to train the policy or fine-tune the base video model.
• In this report, we demonstrate that video models can be used for the entire spectrum of policy evaluation use cases in robotics: from assessing nominal performance to out-of-distribution (OOD) generalization, and probing physical and semantic safety.