Dreaming the Unseen: World Model-regularized Diffusion Policy for Out-of-Distribution Robustness

Understand

Diffusion policies excel at visuomotor control but often fail catastrophically under severe out-of-distribution (OOD) disturbances, such as unexpected object displacements or visual corruptions.

• To address this vulnerability, we introduce the Dream Diffusion Policy (DDP), a framework that deeply integrates a diffusion world model into the policy's training objective via a shared 3D visual encoder.
• This co-optimization endows the policy with robust state-prediction capabilities.
• When encountering sudden OOD anomalies during inference, DDP detects the real-imagination discrepancy and actively abandons the corrupted visual stream.