A Deeper Look at Experience Replay

Understand

Recently experience replay is widely used in various deep reinforcement learning (RL) algorithms, in this paper we rethink the utility of experience replay.

• It introduces a new hyper-parameter, the memory buffer size, which needs carefully tuning.
• However unfortunately the importance of this new hyper-parameter has been underestimated in the community for a long time.
• In this paper we did a systematic empirical study of experience replay under various function representations.