Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modeling

Understand

Videos inherently represent 2D projections of a dynamic 3D world.

• However, our analysis suggests that video diffusion models trained solely on raw video data often fail to capture meaningful geometric-aware structure in their learned representations.
• To bridge the gap between video diffusion models and the underlying 3D nature of the physical world, we propose Geometry Forcing, a simple yet effective method that encourages video diffusion models to internalize 3D representations.
• Our key insight is to guide the model's intermediate representations toward geometry-aware structure by aligning them with features from a geometric foundation model.