Submix: Practical Private Prediction for Large-Scale Language Models

Understand

Recent data-extraction attacks have exposed that language models can memorize some training samples verbatim.

• This is a vulnerability that can compromise the privacy of the model's training data.
• In this work, we introduce SubMix: a practical protocol for private next-token prediction designed to prevent privacy violations by language models that were fine-tuned on a private corpus after pre-training on a public corpus.
• We show that SubMix limits the leakage of information that is unique to any individual user in the private corpus via a relaxation of group differentially private prediction.