Model Imitation for Model-Based Reinforcement Learning

Understand

Model-based reinforcement learning (MBRL) aims to learn a dynamic model to reduce the number of interactions with real-world environments.

• However, due to estimation error, rollouts in the learned model, especially those of long horizons, fail to match the ones in real-world environments.
• This mismatching has seriously impacted the sample complexity of MBRL.
• The phenomenon can be attributed to the fact that previous works employ supervised learning to learn the one-step transition models, which has inherent difficulty ensuring the matching of distributions from multi-step rollouts.