Selective Experience Replay for Lifelong Learning

Understand

Deep reinforcement learning has emerged as a powerful tool for a variety of learning tasks, however deep nets typically exhibit forgetting when learning multiple tasks in sequence.

• To mitigate forgetting, we propose an experience replay process that augments the standard FIFO buffer and selectively stores experiences in a long-term memory.
• We explore four strategies for selecting which experiences will be stored: favoring surprise, favoring reward, matching the global training distribution, and maximizing coverage of the state space.
• We show that distribution matching successfully prevents catastrophic forgetting, and is consistently the best approach on all domains tested.