FreeTraj: Tuning-Free Trajectory Control in Video Diffusion Models

Understand

Diffusion model has demonstrated remarkable capability in video generation, which further sparks interest in introducing trajectory control into the generation process.

• While existing works mainly focus on training-based methods (e.g., conditional adapter), we argue that diffusion model itself allows decent control over the generated content without requiring any training.
• In this study, we introduce a tuning-free framework to achieve trajectory-controllable video generation, by imposing guidance on both noise construction and attention computation.
• Specifically, 1) we first show several instructive phenomenons and analyze how initial noises influence the motion trajectory of generated content.