Unbiased Methods for Multi-Goal Reinforcement Learning

Understand

In multi-goal reinforcement learning (RL) settings, the reward for each goal is sparse, and located in a small neighborhood of the goal.

• In large dimension, the probability of reaching a reward vanishes and the agent receives little learning signal.
• Methods such as Hindsight Experience Replay (HER) tackle this issue by also learning from realized but unplanned-for goals.
• But HER is known to introduce bias, and can converge to low-return policies by overestimating chancy outcomes.