Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Training

Understand

Large language models (LLMs) have become the backbone of modern natural language processing but pose privacy concerns about leaking sensitive training data.

• Membership inference attacks (MIAs), which aim to infer whether a sample is included in a model's training dataset, can serve as a foundation for broader privacy threats.
• Existing defenses designed for traditional classification models do not account for the sequential nature of text data.
• As a result, they either require significant computational resources or fail to effectively mitigate privacy risks in LLMs.