DESIRE: Dynamic Knowledge Consolidation for Rehearsal-Free Continual Learning

Understand

Continual learning aims to equip models with the ability to retain previously learned knowledge like a human.

• Recent work incorporating Parameter-Efficient Fine-Tuning has revitalized the field by introducing lightweight extension modules.
• However, existing methods usually overlook the issue of information leakage caused by the fact that the experiment data have been used in pre-trained models.
• Once these duplicate data are removed in the pre-training phase, their performance can be severely affected.