Mitigating Planner Overfitting in Model-Based Reinforcement Learning

Understand

An agent with an inaccurate model of its environment faces a difficult choice: it can ignore the errors in its model and act in the real world in whatever way it determines is optimal with respect to its model.

• Alternatively, it can take a more conservative stance and eschew its model in favor of optimizing its behavior solely via real-world interaction.
• This latter approach can be exceedingly slow to learn from experience, while the former can lead to "planner overfitting" - aspects of the agent's behavior are optimized to exploit errors in its model.
• This paper explores an intermediate position in which the planner seeks to avoid overfitting through a kind of regularization of the plans it considers.