Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting

Understand

Addressing catastrophic forgetting is one of the key challenges in continual learning where machine learning systems are trained with sequential or streaming tasks.

• Despite recent remarkable progress in state-of-the-art deep learning, deep neural networks (DNNs) are still plagued with the catastrophic forgetting problem.
• This paper presents a conceptually simple yet general and effective framework for handling catastrophic forgetting in continual learning with DNNs.
• The proposed method consists of two components: a neural structure optimization component and a parameter learning and/or fine-tuning component.