SEPS: A Separability Measure for Robust Unlearning in LLMs

Understand

Machine unlearning aims to selectively remove targeted knowledge from Large Language Models (LLMs), ensuring they forget specified content while retaining essential information.

• Existing unlearning metrics assess whether a model correctly answers retain queries and rejects forget queries, but they fail to capture real-world scenarios where forget queries rarely appear in isolation.
• In fact, forget and retain queries often coexist within the same prompt, making mixed-query evaluation crucial.
• We introduce SEPS, an evaluation framework that explicitly measures a model's ability to both forget and retain information within a single prompt.