Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective

Understand

While reinforcement learning (RL) methods that learn an internal model of the environment have the potential to be more sample efficient than their model-free counterparts, learning to model raw observations from high dimensional sensors can be challenging.

• Prior work has addressed this challenge by learning low-dimensional representation of observations through auxiliary objectives, such as reconstruction or value prediction.
• However, the alignment between these auxiliary objectives and the RL objective is often unclear.
• In this work, we propose a single objective which jointly optimizes a latent-space model and policy to achieve high returns while remaining self-consistent.