Demystifying Language Model Forgetting with Low-rank Example Associations

Understand

Large language models (LLMs) suffer from forgetting of upstream knowledge when fine-tuned.

• Despite efforts on mitigating forgetting, few have investigated how forgotten upstream examples are dependent on newly learned tasks.
• Insights on such dependencies enable efficient and targeted mitigation of forgetting.
• In this paper, we empirically analyze forgetting that occurs in $N$ upstream examples of language modeling or instruction-tuning after fine-tuning LLMs on one of $M$ new tasks, visualized in $M\times N$ matrices.