2022

A Systematic Study of Bias Amplification

Hall, Melissa, van der Maaten, Laurens, Gustafson, Laura et al.

Understand

Recent research suggests that predictions made by machine-learning models can amplify biases present in the training data.

When a model amplifies bias, it makes certain predictions at a higher rate for some groups than expected based on training-data statistics.
Mitigating such bias amplification requires a deep understanding of the mechanics in modern machine learning that give rise to that amplification.
We perform the first systematic, controlled study into when and how bias amplification occurs.

Built on

An intersectional definition of fairness. In Proceedings of the International Conference on Data Engineering (ICDE) . 1918–1921
J. Foulds, R. Islam, K.N. Keya, and S. Pan. 2020 · 1921
Earlier work this paper cites.
Estimation of Dependences Based on Empirical Data
V. N. Vapnik. 1982 · 1982
Earlier work this paper cites.
A Method for Solving a Convex Programming Problem with Convergence Rate O(1/K2)
Y. Nesterov. 1983 · 1983
Earlier work this paper cites.
Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods. In Advances in Large Margin Classifiers . MIT Press, 61–74
John C. Platt. 1999 · 1999
Earlier work this paper cites.
Statistical modeling: The two cultures
L. Breiman. 2001 · 2001
Earlier work this paper cites.
Imperfect imaGANation: Implications of GANs exacerbating biases on facial data augmentation and Snapchat selfie lenses. In arXiv preprint arXiv:2001.09528
Original
N. Jain, A. Olmo, S. Sengupta, L. Manikonda, and S. Kambhampati. 2020 · 2001
Earlier work this paper cites.
Theory of classification: a survey of recent advances
S. Boucheron, O. Bousquet, and G. Lugosi. 2005 · 2005
Earlier work this paper cites.
Do instrumental variables belong in propensity scores?
J. Bhattacharya and W.B. Vogt. 2007 · 2007
Earlier work this paper cites.
Learning Multiple Layers of Features from Tiny Images
Alex Krizhevsky. 2009 · 2009
Earlier work this paper cites.
On a class of bias-amplifying variables that endanger effect estimates. In Uncertainty in Artificial Intelligence
J. Pearl. 2010 · 2010
Earlier work this paper cites.
Invited commentary: Understanding bias amplification
J. Pearl. 2011 · 2011
Earlier work this paper cites.
Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference
C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel. 2012 · 2012
Earlier work this paper cites.
Counterfactual Reasoning and Learning Systems: The Example of Computational Advertising
Leon Bottou, Jonas Peters, Joaquin Quinonero-Candela, Denis X. Charles, D. Max Chickering, Elon Portugaly, Dipankar Ray, Patrice Simard, and Ed Snelson. 2013 · 2013
Earlier work this paper cites.
Fair prediction with disparate impact: A study of bias in recidivism prediction instrument
A. Chouldechova. 2016 · 2016
Earlier work this paper cites.
The case for process fairness in learning: Feature selection for fair decision making. In NeurIPS Symposium on Machine Learning and the Law
N. Grgic-Hlaca, M.B. Zafar, K. P. Gummadi, and A. Weller. 2016 · 2016
Earlier work this paper cites.

Similar

Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems (NeurIPS)
M. Hardt, E. Price, and N. Srebro. 2016 · 2016
Cited alongside, same era.
Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016 · 2016
Cited alongside, same era.
Bias amplification and bias unmasking
J.A. Middleton, M.A. Scott, R. Diakow, and J.L. Hill. 2016 · 2016
Cited alongside, same era.
Should instrumental variables be used as matching variables?
J.M. Wooldridge. 2016 · 2016
Cited alongside, same era.
Accurate, large minibatch SGD: training imagenet in 1 hour
Original
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. 2017 · 2017
Cited alongside, same era.
On calibration of modern neural networks. In Proceedings of the International Conference on Machine Learning (ICML) . 1321–1330
C. Guo, G. Pleiss, Y. Sun, and K.Q. Weinberger. 2017 · 2017
Cited alongside, same era.
Counterfactual fairness. In Advances in Neural Information Processing Systems (NeurIPS)
M.J. Kusner, J.R. Loftus, C. Russell, and R. Silva. 2017 · 2017
Cited alongside, same era.
Gender as a variable in natural-language processing: Ethical considerations. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing
B.N. Larson. 2017 · 2017
Cited alongside, same era.
ConvNets and ImageNet Beyond Accuracy: Explanations, Bias Detection, Adversarial Examples and Model Criticism. In arXiv:1711.11443
Original
P. Stock and M. Cisse. 2017 · 2017
Cited alongside, same era.
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017 · 2017
Cited alongside, same era.
Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)
J. Zhao, T. Wang, M. Yatskar, V. Ordonez, and K.-W. Chang. 2017 · 2017
Cited alongside, same era.
Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Fairness, Accountability, and Transparency (FAT)
J. Buolamwini and T. Gebru. 2018 · 2018
Cited alongside, same era.

Then

Calibrated recommendations. In Proceedings of the 12th ACM conference on recommender systems . 154–162
Harald Steck. 2018 · 2018
Later among the works it cites.
Does Object Recognition Work for Everyone?. In CVPR Workshop on Computer Vision for Global Challenges
T. DeVries, I. Misra, C. Wang, and L.J.P. van der Maaten. 2019 · 2019
Later among the works it cites.
Feature-wise bias amplification. In Proceedings of the International Conference on Learning Representations (ICLR)
K. Leino, E. Black, M. Fredrikson, S. Sen, and A. Datta. 2019 · 2019
Later among the works it cites.
Dissecting racial bias in an algorithm used to manage the health of populations
Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019 · 2019
Later among the works it cites.
Fair generative modeling via weak supervision. In Proceedings of the International Conference on Machine Learning (ICML)
K. Choi, A. Grover, T. Singh, R. Shu, and S. Ermon. 2020 · 2020
Later among the works it cites.
Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)
Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela, and Jason Weston. 2020 · 2020
Later among the works it cites.
Mitigating gender bias amplification in distribution by posterior regularization. In Annual Meeting of the Association for Computational Linguistics (ACL)
S. Jia, T. Meng, J. Zhao, and K.-W Chang. 2020 · 2020
Later among the works it cites.
How we’ve taught algorithms to see identity: Constructing race and gender in image databases for facial analysis. In Proceedings of the ACM on Human-Computer Interaction
M. K. Scheuerman, K. Wade, C. Lustig, and J. R. Brubaker. 2020 · 2020
Later among the works it cites.
Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production Systems
Original
Chloé Bakalar, Renata Barreto, Stevie Bergman, Miranda Bogen, Bobbie Chern, Sam Corbett-Davies, Melissa Hall, Isabel Kloumann, Michelle Lam, Joaquin Quiñonero Candela, et al · 2021
Later among the works it cites.
Fairness in criminal justice risk assessments: The state of the art
R. Berk, H. Heidari, S. Jabbari, M. Kearns, and A. Roth. 2021 · 2021
Later among the works it cites.
Moving beyond “algorithmic bias is a data problem”
S. Hooker. 2021 · 2021
Later among the works it cites.
Learning Transferable Visual Models From Natural Language Supervision. In arXiv:2103.00020
Original
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021 · 2021
Later among the works it cites.
Directional Bias Amplification. In Proceedings of the International Conference on Machine Learning (ICML)
Angelina Wang and Olga Russakovsky. 2021 · 2021
Later among the works it cites.
Understanding and Evaluating Racial Biases in Image Captioning. In arXiv preprint arXiv:2106.08503
Original
D. Zhao, A. Wang, and O. Russakovsky. 2021 · 2021
Later among the works it cites.
A survey on bias and fairness in machine learning
N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan. 2022 · 2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…