Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond

Understand

Large language models (LLMs) should undergo rigorous audits to identify potential risks, such as copyright and privacy infringements.

• Once these risks emerge, timely updates are crucial to remove undesirable responses, ensuring legal and safe model usage.
• It has spurred recent research into LLM unlearning, focusing on erasing targeted undesirable knowledge without compromising the integrity of other, non-targeted responses.
• Existing studies have introduced various unlearning objectives to pursue LLM unlearning without necessitating complete retraining.