MADE: Exploration via Maximizing Deviation from Explored Regions

Understand

In online reinforcement learning (RL), efficient exploration remains particularly challenging in high-dimensional environments with sparse rewards.

• In low-dimensional environments, where tabular parameterization is possible, count-based upper confidence bound (UCB) exploration methods achieve minimax near-optimal rates.
• However, it remains unclear how to efficiently implement UCB in realistic RL tasks that involve non-linear function approximation.
• To address this, we propose a new exploration approach via \textit{maximizing} the deviation of the occupancy of the next policy from the explored regions.