Semantic Residual Prompts for Continual Learning

Understand

Prompt-tuning methods for Continual Learning (CL) freeze a large pre-trained model and train a few parameter vectors termed prompts.

• Most of these methods organize these vectors in a pool of key-value pairs and use the input image as query to retrieve the prompts (values).
• However, as keys are learned while tasks progress, the prompting selection strategy is itself subject to catastrophic forgetting, an issue often overlooked by existing approaches.
• For instance, prompts introduced to accommodate new tasks might end up interfering with previously learned prompts.