RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments

Understand

Exploration in sparse reward environments remains one of the key challenges of model-free reinforcement learning.

• Instead of solely relying on extrinsic rewards provided by the environment, many state-of-the-art methods use intrinsic rewards to encourage exploration.
• However, we show that existing methods fall short in procedurally-generated environments where an agent is unlikely to visit a state more than once.
• We propose a novel type of intrinsic reward which encourages the agent to take actions that lead to significant changes in its learned state representation.