Adaptive Differentially Private Empirical Risk Minimization

Understand

We propose an adaptive (stochastic) gradient perturbation method for differentially private empirical risk minimization.

• At each iteration, the random noise added to the gradient is optimally adapted to the stepsize; we name this process adaptive differentially private (ADP) learning.
• Given the same privacy budget, we prove that the ADP method considerably improves the utility guarantee compared to the standard differentially private method in which vanilla random noise is added.
• Our method is particularly useful for gradient-based algorithms with time-varying learning rates, including variants of AdaGrad (Duchi et al., 2011).