Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learning

Understand

A fundamental challenge in reinforcement learning is to learn policies that generalize beyond the operating domains experienced during training.

• In this paper, we approach this challenge through the following invariance principle: an agent must find a representation such that there exists an action-predictor built on top of this representation that is simultaneously optimal across all training domains.
• Intuitively, the resulting invariant policy enhances generalization by finding causes of successful actions.
• We propose a novel learning algorithm, Invariant Policy Optimization (IPO), that implements this principle and learns an invariant policy during training.