Animate Your Motion: Turning Still Images into Dynamic Videos

Understand

In recent years, diffusion models have made remarkable strides in text-to-video generation, sparking a quest for enhanced control over video outputs to more accurately reflect user intentions.

• Traditional efforts predominantly focus on employing either semantic cues, like images or depth maps, or motion-based conditions, like moving sketches or object bounding boxes.
• Semantic inputs offer a rich scene context but lack detailed motion specificity; conversely, motion inputs provide precise trajectory information but miss the broader semantic narrative.
• For the first time, we integrate both semantic and motion cues within a diffusion model for video generation, as demonstrated in Fig 1.