LoLA: Long Horizon Latent Action Learning for General Robot Manipulation

Understand

The capability of performing long-horizon, language-guided robotic manipulation tasks critically relies on leveraging historical information and generating coherent action sequences.

• However, such capabilities are often overlooked by existing Vision-Language-Action (VLA) models.
• To solve this challenge, we propose LoLA (Long Horizon Latent Action Learning), a framework designed for robot manipulation that integrates long-term multi-view observations and robot proprioception to enable multi-step reasoning and action generation.
• We first employ Vision-Language Models to encode rich contextual features from historical sequences and multi-view observations.