VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation

Understand

Generative models that can model and predict sequences of future events can, in principle, learn to capture complex real-world phenomena, such as physical interactions.

• However, a central challenge in video prediction is that the future is highly uncertain: a sequence of past observations of events can imply many possible futures.
• Although a number of recent works have studied probabilistic models that can represent uncertain futures, such models are either extremely expensive computationally as in the case of pixel-level autoregressive models, or do not directly optimize the likelihood of the data.
• To our knowledge, our work is the first to propose multi-frame video prediction with normalizing flows, which allows for direct optimization of the data likelihood, and produces high-quality stochastic predictions.