Towards Incremental Learning in Large Language Models: A Critical Review

Understand

Incremental learning is the ability of systems to acquire knowledge over time, enabling their adaptation and generalization to novel tasks.

• It is a critical ability for intelligent, real-world systems, especially when data changes frequently or is limited.
• This review provides a comprehensive analysis of incremental learning in Large Language Models.
• It synthesizes the state-of-the-art incremental learning paradigms, including continual learning, meta-learning, parameter-efficient learning, and mixture-of-experts learning.