Training Data Influence Analysis and Estimation: A Survey

Understand

Good models require good training data.

• For overparameterized deep models, the causal relationship between training data and model predictions is increasingly opaque and poorly understood.
• Influence analysis partially demystifies training's underlying interactions by quantifying the amount each training instance alters the final model.
• Measuring the training data's influence exactly can be provably hard in the worst case; this has led to the development and use of influence estimators, which only approximate the true influence.