MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusion

Understand

We introduce Multi-view Ancestral Sampling (MAS), a method for 3D motion generation, using 2D diffusion models that were trained on motions obtained from in-the-wild videos.

• As such, MAS opens opportunities to exciting and diverse fields of motion previously under-explored as 3D data is scarce and hard to collect.
• MAS works by simultaneously denoising multiple 2D motion sequences representing different views of the same 3D motion.
• It ensures consistency across all views at each diffusion step by combining the individual generations into a unified 3D sequence, and projecting it back to the original views.