Differentially Private Learning with Per-Sample Adaptive Clipping

Understand

Privacy in AI remains a topic that draws attention from researchers and the general public in recent years.

• As one way to implement privacy-preserving AI, differentially private learning is a framework that enables AI models to use differential privacy (DP).
• To achieve DP in the learning process, existing algorithms typically limit the magnitude of gradients with a constant clipping, which requires carefully tuned due to its significant impact on model performance.
• As a solution to this issue, latest works NSGD and Auto-S innovatively propose to use normalization instead of clipping to avoid hyperparameter tuning.