Undesirable Memorization in Large Language Models: A Survey

Understand

While recent research increasingly showcases the remarkable capabilities of Large Language Models (LLMs), it is equally crucial to examine their associated risks.

• Among these, privacy and security vulnerabilities are particularly concerning, posing significant ethical and legal challenges.
• At the heart of these vulnerabilities stands memorization, which refers to a model's tendency to store and reproduce phrases from its training data.
• This phenomenon has been shown to be a fundamental source to various privacy and security attacks against LLMs.