Toward Training at ImageNet Scale with Differential Privacy

Understand

Differential privacy (DP) is the de facto standard for training machine learning (ML) models, including neural networks, while ensuring the privacy of individual examples in the training set.

• Despite a rich literature on how to train ML models with differential privacy, it remains extremely challenging to train real-life, large neural networks with both reasonable accuracy and privacy.
• We set out to investigate how to do this, using ImageNet image classification as a poster example of an ML task that is very challenging to resolve accurately with DP right now.
• This paper shares initial lessons from our effort, in the hope that it will inspire and inform other researchers to explore DP training at scale.