The Effects of Memory Replay in Reinforcement Learning

Understand

Experience replay is a key technique behind many recent advances in deep reinforcement learning.

• Allowing the agent to learn from earlier memories can speed up learning and break undesirable temporal correlations.
• Despite its wide-spread application, very little is understood about the properties of experience replay.
• How does the amount of memory kept affect learning dynamics? Does it help to prioritize certain experiences? In this paper, we address these questions by formulating a dynamical systems ODE model of Q-learning with experience replay.