Hierarchical Video Generation from Orthogonal Information: Optical Flow and Texture

Understand

Learning to represent and generate videos from unlabeled data is a very challenging problem.

• To generate realistic videos, it is important not only to ensure that the appearance of each frame is real, but also to ensure the plausibility of a video motion and consistency of a video appearance in the time direction.
• The process of video generation should be divided according to these intrinsic difficulties.
• In this study, we focus on the motion and appearance information as two important orthogonal components of a video, and propose Flow-and-Texture-Generative Adversarial Networks (FTGAN) consisting of FlowGAN and TextureGAN.