Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environments

Understand

Exploration under sparse reward is a long-standing challenge of model-free reinforcement learning.

• The state-of-the-art methods address this challenge by introducing intrinsic rewards to encourage exploration in novel states or uncertain environment dynamics.
• Unfortunately, methods based on intrinsic rewards often fall short in procedurally-generated environments, where a different environment is generated in each episode so that the agent is not likely to visit the same state more than once.
• Motivated by how humans distinguish good exploration behaviors by looking into the entire episode, we introduce RAPID, a simple yet effective episode-level exploration method for procedurally-generated environments.