Differentially Private Learning Needs Better Features (or Much More Data)

Understand

We demonstrate that differentially private machine learning has not yet reached its "AlexNet moment" on many canonical vision tasks: linear models trained on handcrafted features significantly outperform end-to-end deep neural networks for moderate privacy budgets.

• To exceed the performance of handcrafted features, we show that private learning requires either much more private data, or access to features learned on public data from a similar domain.
• Our work introduces simple yet strong baselines for differentially private learning that can inform the evaluation of future progress in this area.