Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Data

Understand

We consider the problem of a training data proof, where a data creator or owner wants to demonstrate to a third party that some machine learning model was trained on their data.

• Training data proofs play a key role in recent lawsuits against foundation models trained on web-scale data.
• Many prior works suggest to instantiate training data proofs using membership inference attacks.
• We argue that this approach is fundamentally unsound: to provide convincing evidence, the data creator needs to demonstrate that their attack has a low false positive rate, i.e., that the attack's output is unlikely under the null hypothesis that the model was not trained on the target data.