Experience Replay for Continual Learning

Understand

Continual learning is the problem of learning new tasks or knowledge while protecting old knowledge and ideally generalizing from old experience to learn new tasks faster.

• Neural networks trained by stochastic gradient descent often degrade on old tasks when trained successively on new tasks with different data distributions.
• This phenomenon, referred to as catastrophic forgetting, is considered a major hurdle to learning with non-stationary data or sequences of new tasks, and prevents networks from continually accumulating knowledge and skills.
• We examine this issue in the context of reinforcement learning, in a setting where an agent is exposed to tasks in a sequence.