Measuring Forgetting of Memorized Training Examples

Understand

Machine learning models exhibit two seemingly contradictory phenomena: training data memorization, and various forms of forgetting.

• In memorization, models overfit specific training examples and become susceptible to privacy attacks.
• In forgetting, examples which appeared early in training are forgotten by the end.
• In this work, we connect these phenomena.