2022

Disparate Impact in Differential Privacy from Gradient Misalignment

Esipova, Maria S., Ghomi, Atiyeh Ashari, Luo, Yaqiao et al.

Understand

As machine learning becomes more widespread throughout society, aspects including data privacy and fairness must be carefully considered, and are crucial for deployment in highly regulated industries.

Unfortunately, the application of privacy enhancing technologies can worsen unfair tendencies in models.
In particular, one of the most widely used techniques for private model training, differentially private stochastic gradient descent (DPSGD), frequently intensifies disparate impact on groups within data.
In this work we study the fine-grained causes of unfairness in DPSGD and identify gradient misalignment due to inequitable gradient clipping as the most significant source.

Built on

How to generate and exchange secrets
Andrew Chi-Chih Yao · 1986
Earlier work this paper cites.
A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines
M.F. Hutchinson · 1990
Earlier work this paper cites.
The 2001 Census in the Netherlands Integration of Registers and Surveys
Paul van der Laan · 2000
Earlier work this paper cites.
Calibrating Noise to Sensitivity in Private Data Analysis
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith · 2006
Earlier work this paper cites.
Fully Homomorphic Encryption Using Ideal Lattices
Craig Gentry · 2009
Earlier work this paper cites.
Differentially Private Empirical Risk Minimization
Kamalika Chaudhuri, Claire Monteleoni, and Anand D. Sarwate · 2011
Earlier work this paper cites.
The Algorithmic Foundations of Differential Privacy
Cynthia Dwork and Aaron Roth · 2014
Earlier work this paper cites.
Deep learning face attributes in the wild
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang · 2015
Earlier work this paper cites.
Deep learning with differential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang · 2016
Earlier work this paper cites.
Communication-Efficient Learning of Deep Networks from Decentralized Data
H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas · 2017
Earlier work this paper cites.
Rényi differential privacy
Ilya Mironov · 2017
Earlier work this paper cites.
The U.S. Census Bureau Adopts Differential Privacy
John M. Abowd · 2018
Earlier work this paper cites.
Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification
Joy Buolamwini and Timnit Gebru · 2018
Earlier work this paper cites.

Similar

Privacy for All: Ensuring Fair and Equitable Privacy Protections
Michael D. Ekstrand, Rezvan Joshaghani, and Hoda Mehrpouyan · 2018
Cited alongside, same era.
Differential privacy has disparate impact on model accuracy
Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov · 2019
Cited alongside, same era.
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song · 2019
Cited alongside, same era.
On the Compatibility of Privacy and Fairness
Rachel Cummings, Varun Gupta, Dhamma Kimpara, and Jamie Morgenstern · 2019
Cited alongside, same era.
Differentially Private Fair Learning
Matthew Jagielski, Michael Kearns, Jieming Mao, Alina Oprea, Aaron Roth, Saeed Sharifi Malvajerdi, and Jonathan Ullman · 2019
Cited alongside, same era.
Rényi Differential Privacy of the Sampled Gaussian Mechanism
Original
Ilya Mironov, Kunal Talwar, and Li Zhang · 2019
Cited alongside, same era.
AdaCliP: Adaptive Clipping for Private SGD
Original
Venkatadheeraj Pichapati, Ananda Theertha Suresh, Felix X. Yu, Sashank J. Reddi, and Sanjiv Kumar · 2019
Cited alongside, same era.
Achieving differential privacy and fairness in logistic regression
Depeng Xu, Shuhan Yuan, and Xintao Wu · 2019
Cited alongside, same era.
Understanding Gradient Clipping in Private SGD: A Geometric Perspective
Xiangyi Chen, Steven Z. Wu, and Mingyi Hong · 2020
Cited alongside, same era.
CaPC Learning: Confidential and Private Collaborative Learning
Christopher A Choquette-Choo, Natalie Dullerud, Adam Dziedzic, Yunxiang Zhang, Somesh Jha, Nicolas Papernot, and Xiao Wang · 2020
Cited alongside, same era.
A Snapshot of the Frontiers of Fairness in Machine Learning
Alexandra Chouldechova and Aaron Roth · 2020
Cited alongside, same era.
Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacy
Tom Farrand, Fatemehsadat Mireshghallah, Sahib Singh, and Andrew Trask · 2020
Cited alongside, same era.

Then

Privacy Enhanced Multimodal Neural Representations for Emotion Recognition
Mimansa Jaiswal and Emily Mower Provost · 2020
Later among the works it cites.
Fair learning with private demographic data
Hussein Mozannar, Mesrob Ohannessian, and Nathan Srebro · 2020
Later among the works it cites.
Fair decision making using privacy-protected data
David Pujol, Ryan McKenna, Satya Kuppam, Michael Hay, Ashwin Machanavajjhala, and Gerome Miklau · 2020
Later among the works it cites.
Differentially Private Learning with Adaptive Clipping
Galen Andrew, Om Thakkar, Brendan McMahan, and Swaroop Ramaswamy · 2021
Later among the works it cites.
On the Convergence of Deep Learning with Differential Privacy
Original
Zhiqi Bu, Hua Wang, Qi Long, and Weijie J. Su · 2021
Later among the works it cites.
On the privacy risks of algorithmic fairness
Hongyan Chang and Reza Shokri · 2021
Later among the works it cites.
ProxyFL: Decentralized Federated Learning through Proxy Model Sharing
Original
Shivam Kalra, Junfeng Wen, Jesse C. Cresswell, Maksims Volkovs, and Hamid R. Tizhoosh · 2021
Later among the works it cites.
A Survey on Bias and Fairness in Machine Learning
Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan · 2021
Later among the works it cites.
Adversary instantiation: Lower bounds for differentially private machine learning
Milad Nasr, Shuang Songi, Abhradeep Thakurta, Nicolas Papemoti, and Nicholas Carlin · 2021
Later among the works it cites.
Removing Disparate Impact on Model Accuracy in Differentially Private Stochastic Gradient Descent
Depeng Xu, Wei Du, and Xintao Wu · 2021
Later among the works it cites.
Federated learning and differential privacy for medical image analysis
Mohammed Adnan, Shivam Kalra, Jesse C. Cresswell, Graham W. Taylor, and Hamid R. Tizhoosh · 2022
Closest in time.
A survey on datasets for fairness-aware machine learning
Tai Le Quy, Arjun Roy, Vasileios Iosifidis, Wenbin Zhang, and Eirini Ntoutsi · 2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…