OG-VLA: Orthographic Image Generation for 3D-Aware Vision-Language Action Model

Understand

We introduce OG-VLA, a novel architecture and learning framework that combines the generalization strengths of Vision Language Action models (VLAs) with the robustness of 3D-aware policies.

• We address the challenge of mapping natural language instructions and one or more RGBD observations to quasi-static robot actions.
• 3D-aware robot policies achieve state-of-the-art performance on precise robot manipulation tasks, but struggle with generalization to unseen instructions, scenes, and objects.
• On the other hand, VLAs excel at generalizing across instructions and scenes, but can be sensitive to camera and robot pose variations.