PixelVLA: Advancing Pixel-level Understanding in Vision-Language-Action Model

Understand

Vision-Language-Action models (VLAs) are emerging as powerful tools for learning generalizable visuomotor control policies.

• However, current VLAs are mostly trained on large-scale image-text-action data and remain limited in two key ways: (i) they struggle with pixel-level scene understanding, and (ii) they rely heavily on textual prompts, which reduces their flexibility in real-world settings.
• To address these challenges, we introduce PixelVLA, the first VLA model designed to support both pixel-level reasoning and multimodal prompting with text and visual inputs.
• Our approach is built on a new visuomotor instruction tuning framework that integrates a multiscale pixel-aware encoder with a visual promptaware encoder.