Which priors matter? Benchmarking models for learning latent dynamics

Understand

Learning dynamics is at the heart of many important applications of machine learning (ML), such as robotics and autonomous driving.

• In these settings, ML algorithms typically need to reason about a physical system using high dimensional observations, such as images, without access to the underlying state.
• Recently, several methods have proposed to integrate priors from classical mechanics into ML models to address the challenge of physical reasoning from images.
• In this work, we take a sober look at the current capabilities of these models.