Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model

Understand

Augmenting vision-language-action models (VLAs) with world models is promising for robotic policy learning but faces challenges in jointly predicting states and actions due to the modality gap.

• To address this, we propose DUal-STream diffusion (DUST), a world-model augmented VLA framework featuring a multimodal diffusion transformer that maintains separate modality streams while enabling cross-modal knowledge sharing.
• In addition, DUST utilizes independent noise perturbations and a decoupled flow matching loss to learn cross-modal causal relationships.
• We further introduce an asynchronous sampling method for action and vision tokens that enhances performance through inference-time scaling.