Motion-Conditioned Diffusion Model for Controllable Video Synthesis

Understand

Recent advancements in diffusion models have greatly improved the quality and diversity of synthesized content.

• To harness the expressive power of diffusion models, researchers have explored various controllable mechanisms that allow users to intuitively guide the content synthesis process.
• Although the latest efforts have primarily focused on video synthesis, there has been a lack of effective methods for controlling and describing desired content and motion.
• In response to this gap, we introduce MCDiff, a conditional diffusion model that generates a video from a starting image frame and a set of strokes, which allow users to specify the intended content and dynamics for synthesis.