Latent Policy Steering with Embodiment-Agnostic Pretrained World Models

Understand

The performance of learned robot visuomotor policies is heavily dependent on the size and quality of the training dataset.

• Although large-scale robot and human datasets are increasingly available, embodiment gaps and mismatched action spaces make them difficult to leverage.
• Our main insight is that skills performed across different embodiments produce visual similarities in motions that can be captured using off-the-shelf action representations such as optical flow.
• Moreover, World Models (WMs) can leverage sub-optimal data since they focus on modeling dynamics.