Experience Replay Optimization

Understand

Experience replay enables reinforcement learning agents to memorize and reuse past experiences, just as humans replay memories for the situation at hand.

• Contemporary off-policy algorithms either replay past experiences uniformly or utilize a rule-based replay strategy, which may be sub-optimal.
• In this work, we consider learning a replay policy to optimize the cumulative reward.
• Replay learning is challenging because the replay memory is noisy and large, and the cumulative reward is unstable.