Animating Landscape: Self-Supervised Learning of Decoupled Motion and Appearance for Single-Image Video Synthesis

Understand

Automatic generation of a high-quality video from a single image remains a challenging task despite the recent advances in deep generative models.

• This paper proposes a method that can create a high-resolution, long-term animation using convolutional neural networks (CNNs) from a single landscape image where we mainly focus on skies and waters.
• Our key observation is that the motion (e.g., moving clouds) and appearance (e.g., time-varying colors in the sky) in natural scenes have different time scales.
• We thus learn them separately and predict them with decoupled control while handling future uncertainty in both predictions by introducing latent codes.