Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions

Understand

Continual learning is an emerging paradigm in machine learning, wherein a model is exposed in an online fashion to data from multiple different distributions (i.e.

• environments), and is expected to adapt to the distribution change.
• Precisely, the goal is to perform well in the new environment, while simultaneously retaining the performance on the previous environments (i.e.
• avoid "catastrophic forgetting") -- without increasing the size of the model.