Data Portraits: Recording Foundation Model Training Data

Understand

Foundation models are trained on increasingly immense and opaque datasets.

• Even while these models are now key in AI system building, it can be difficult to answer the straightforward question: has the model already encountered a given example during training? We therefore propose a widespread adoption of Data Portraits: artifacts that record training data and allow for downstream inspection.
• First we outline the properties of such an artifact and discuss how existing solutions can be used to increase transparency.
• We then propose and implement a solution based on data sketching, stressing fast and space efficient querying.