Motus: A Unified Latent Action World Model

Understand

While a general embodied agent must function as a unified system, current methods are built on isolated models for understanding, world modeling, and control.

• This fragmentation prevents unifying multimodal generative capabilities and hinders learning from large-scale, heterogeneous data.
• In this paper, we propose Motus, a unified latent action world model that leverages existing general pretrained models and rich, sharable motion information.
• Motus introduces a Mixture-of-Transformer (MoT) architecture to integrate three experts (i.e., understanding, video generation, and action) and adopts a UniDiffuser-style scheduler to enable flexible switching between different modeling modes (i.e., world models, vision-language-action models, inverse dynamics models, video generation models, and video-action joint prediction models).