VideoPanda: Video Panoramic Diffusion with Multi-view Attention

Understand

High resolution panoramic video content is paramount for immersive experiences in Virtual Reality, but is non-trivial to collect as it requires specialized equipment and intricate camera setups.

• In this work, we introduce VideoPanda, a novel approach for synthesizing 360$^\circ$ videos conditioned on text or single-view video data.
• VideoPanda leverages multi-view attention layers to augment a video diffusion model, enabling it to generate consistent multi-view videos that can be combined into immersive panoramic content.
• VideoPanda is trained jointly using two conditions: text-only and single-view video, and supports autoregressive generation of long-videos.