Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy

Understand

Deep Learning models have achieved remarkable performance in tasks such as image classification or generation, often surpassing human accuracy.

• However, they can struggle to learn new tasks and update their knowledge without access to previous data, leading to a significant loss of accuracy known as Catastrophic Forgetting (CF).
• This phenomenon was first observed by McCloskey and Cohen in 1989 and remains an active research topic.
• Incremental learning without forgetting is widely recognized as a crucial aspect in building better AI systems, as it allows models to adapt to new tasks without losing the ability to perform previously learned ones.