CoMA: Compositional Human Motion Generation with Multi-modal Agents

Understand

3D human motion generation has seen substantial advancement in recent years.

• While state-of-the-art approaches have improved performance significantly, they still struggle with complex and detailed motions unseen in training data, largely due to the scarcity of motion datasets and the prohibitive cost of generating new training examples.
• To address these challenges, we introduce CoMA, an agent-based solution for complex human motion generation, editing, and comprehension.
• CoMA leverages multiple collaborative agents powered by large language and vision models, alongside a mask transformer-based motion generator featuring body part-specific encoders and codebooks for fine-grained control.