Hindsight for Foresight: Unsupervised Structured Dynamics Models from Physical Interaction

Understand

A key challenge for an agent learning to interact with the world is to reason about physical properties of objects and to foresee their dynamics under the effect of applied forces.

• In order to scale learning through interaction to many objects and scenes, robots should be able to improve their own performance from real-world experience without requiring human supervision.
• To this end, we propose a novel approach for modeling the dynamics of a robot's interactions directly from unlabeled 3D point clouds and images.
• Unlike previous approaches, our method does not require ground-truth data associations provided by a tracker or any pre-trained perception network.