QDepth-VLA: Quantized Depth Prediction as Auxiliary Supervision for Vision-Language-Action Models

Understand

Spatial perception and reasoning are crucial for Vision-Language-Action (VLA) models to accomplish fine-grained manipulation tasks.

• However, existing approaches often lack the ability to understand and reason over the essential 3D structures necessary for precise control.
• To address this limitation, we propose QDepth-VLA, a general framework that augments VLA models with an auxiliary depth prediction task.
• A dedicated depth expert is designed to predict quantized latent tokens of depth maps obtained from a VQ-VAE encoder, enabling the model to learn depth-aware representations that capture critical geometric cues.