2020

REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets

Wang, Angelina, Liu, Alexander, Zhang, Ryan et al.

Understand

Machine learning models are known to perpetuate and even amplify the biases present in the data.

However, these data biases frequently do not become apparent until after the models are deployed.
Our work tackles this issue and enables the preemptive analysis of large-scale datasets.
REVISE (REvealing VIsual biaSEs) is a tool that assists in the investigation of a visual dataset, surfacing potential biases along three dimensions: (1) object-based, (2) person-based, and (3) geography-based.

Built on

“Imbalance Problems in Object Detection: A Review”
Original
Kemal Oksuz, Baris Cam, Sinan Kalkan and Emre Akbas · 1909
Earlier work this paper cites.
“A Distribution-Free k-Sample Test Against Ordered Alternatives”
Aimable Jonckheere · 1954
Earlier work this paper cites.
“80 million tiny images: a large dataset for non-parametric object and scene recognition”
Antonio Torralba, Rob Fergus and William. Freeman · 1970
Earlier work this paper cites.
“On the optimal number of features in the classification of multivariate gaussian data”
Anil. Jain and W.G. Waller · 1978
Earlier work this paper cites.
“The myth of the neutral ’man”’
Janice Moulton · 1981
Earlier work this paper cites.
“The validity and practicality of sun- reactive skin types I through VI”
Thomas Fitzpatrick · 1988
Earlier work this paper cites.
“Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories”
Li Fei-Fei, Rob Fergus and Pietro Perona · 2004
Earlier work this paper cites.
“Optimal number of features as a function of sample size for various classification rules”
Jianping Hua et al · 2005
Earlier work this paper cites.
“The role of context in object recognition”
Aude Oliva and Antonio Torralba · 2007
Earlier work this paper cites.
“Object Categorization using Co-Occurrence, Location and Appearance”
Carolina Galleguillos, Andrew Rabinovich and Serge Belongie · 2008
Earlier work this paper cites.
“The Fabric of Internalized Sexism”
Steve Bearman, Neill Korobov and Avril Thorne · 2009
Earlier work this paper cites.
“ImageNet: A Large-Scale Hierarchical Image Database”
J. Deng et al · 2009
Earlier work this paper cites.
“Learning Multiple Layers of Features from Tiny Images”
Alex Krizhevsky · 2009
Earlier work this paper cites.
“Exploratory Undersampling for Class-Imbalance Learning”
Xu-Ying Liu, Jianxin Wu and Zhi-Hua Zhou · 2009
Earlier work this paper cites.
“The PASCAL Visual Object Classes (VOC) Challenge”
Mark Everingham et al · 2010
Earlier work this paper cites.
“SUN Database: Large-scale Scene Recognition from Abbey to Zoo”
Jianxiong Xiao et al · 2010
Earlier work this paper cites.
“Learning to Share Visual Appearance for Multiclass Object Detection”
Ruslan Salakhutdinov, Antonio Torralba and Josh Tenenbaum · 2011
Earlier work this paper cites.
“Unbiased Look at Dataset Bias”
Antonio Torralba and Alexei. Efros · 2011
Earlier work this paper cites.
“Understanding and Predicting Importance in Images”
Alexander. Berg et al · 2012
Earlier work this paper cites.
“Context Models and Out-of-context Objects”
Myung Choi, Antonio Torralba and Alan. Willsky · 2012
Earlier work this paper cites.
“Fairness Through Awareness”
Cynthia Dwork et al · 2012
Earlier work this paper cites.
“Diagnosing Error in Object Detectors”
Derek Hoiem, Yodsawalai Chodpathumwan and Qieyun Dai · 2012
Earlier work this paper cites.
“Undoing the damage of dataset bias”
Aditya Khosla et al · 2012
Earlier work this paper cites.
“ImageNet Classification with Deep Convolutional Neural Networks”
Alex Krizhevsky, Ilya Sutskever and Geoffrey. Hinton · 2012
Earlier work this paper cites.
“Archives and Recordkeeping: Theory into Practices”
Caroline Brown · 2014
Earlier work this paper cites.
“Microsoft COCO: Common Objects in Context”
Tsung-Yi Lin et al · 2014
Earlier work this paper cites.
“Context driven scene parsing with attention to rare classes”
Jimei Yang, Brian Price, Scott Cohen and Ming-Hsuan Yang · 2014
Earlier work this paper cites.
“Capturing long-tail distributions of object subcategories”
Xiangxin Zhu, Dragomir Anguelov and Deva Ramanan · 2014
Earlier work this paper cites.
“Unequal Representation and Gender Stereotypes in Image Search Results for Occupations”
Matthew Kay, Cynthia Matuszek and Sean. Munson · 2015
Earlier work this paper cites.
“ImageNet Large Scale Visual Recognition Challenge”
Olga Russakovsky et al · 2015
Earlier work this paper cites.
“A Deeper Look at Dataset Bias”
Tatiana Tommasi, Novi Patricia, Barbara Caputo and Tinne Tuytelaars · 2015
Earlier work this paper cites.

Similar

“Equality of Opportunity in Supervised Learning”
Moritz Hardt, Eric Price and Nathan Srebro · 2016
Cited alongside, same era.
“Deep Residual Learning for Image Recognition”
Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun · 2016
Cited alongside, same era.
“FastText.zip: Compressing text classification models”
Original
Armand Joulin et al · 2016
Cited alongside, same era.
“Bag of Tricks for Efficient Text Classification”
Original
Armand Joulin, Edouard Grave, Piotr Bojanowski and Tomas Mikolov · 2016
Cited alongside, same era.
“Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations”, 2016
Original
Ranjay Krishna et al · 2016
Cited alongside, same era.
“Factors in Finetuning Deep Model for Object Detection with Long-tail Distribution”
Wanli Ouyang, Xiaogang Wang, Cong Zhang and Xiaokang Yang · 2016
Cited alongside, same era.
“YFCC100M: The New Data in Multimedia Research”
Bart Thomee et al · 2016
Cited alongside, same era.
“A systematic study of the class imbalance problem in convolutional neural networks”
Original
Mateusz Buda, Atsuto Maki and Maciej. Mazurowski · 2017
Cited alongside, same era.
“Semantics derived automatically from language corpora contain humanlike biases”
Aylin Caliskan, Joanna. Bryson and Arvind Narayanan · 2017
Cited alongside, same era.
“Fair prediction with disparate impact: A study of bias in recidivism prediction instruments”
Alexandra Chouldechova · 2017
Cited alongside, same era.
“Decoupled classifiers for fair and efficient machine learning”
Original
Cynthia Dwork, Nicole Immorlica, Adam Kalai and Max Leiserson · 2017
Cited alongside, same era.
“On formalizing fairness in prediction with machine learning”
Original
Pratik Gajane and Mykola Pechenizkiy · 2017
Cited alongside, same era.

Then

“The Elephant in the Room”
Original
Amir Rosenfeld, Richard Zemel and John. Tsotsos · 2018
Later among the works it cites.
“Mitigating unwanted biases with adversarial learning”
Brian Zhang, Blake Lemoine and Margaret Mitchell · 2018
Later among the works it cites.
“Fairness and Machine Learning” http://www.fairmlbook.org
Solon Barocas, Moritz Hardt and Arvind Narayanan · 2019
Later among the works it cites.
“Does Object Recognition Work for Everyone?”
Terrance DeVries, Ishan Misra, Changhan Wang and Laurens van Maaten · 2019
Later among the works it cites.
URL: https://github.com/akamaster/pytorch_resnet_cifar10
Yerlan Idelbayev, 2019 · 2019
Later among the works it cites.
“Median Incomes”
Keeping Track Online · 2019
Later among the works it cites.
“A Survey on Bias and Fairness in Machine Learning”
Original
Ninareh Mehrabi et al · 2019
Later among the works it cites.
“What are the biases in my word embedding?”
Nathaniel Swinger et al · 2019
Later among the works it cites.
“American Community Survey 1-Year Estimates, Table S1903 (2005-2019)”
The United States Census Bureau · 2019
Later among the works it cites.
“United Nations Statistics Division - Methodology”, 2019
United Nations Statistics Division · 2019
Later among the works it cites.
“Predictive Inequity in Object Detection”
Original
Benjamin Wilson, Judy Hoffman and Jamie Morgenstern · 2019
Later among the works it cites.
“SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognition”
Kaiyu Yang, Olga Russakovsky and Jia Deng · 2019
Later among the works it cites.
“Bringing the People Back In: Contesting Benchmark Machine Learning Datasets”
Original
Emily Denton et al · 2020
Closest in time.
“Towards a Critical Race Methodology in Algorithmic Fairness”
Alex Hanna, Emily Denton, Andrew Smart and Jamila Smith-Loud · 2020
Closest in time.
“Wrongfully Accused by an Algorithm”
Kashmir Hill · 2020
Closest in time.
“spaCy: Industrial-strength Natural Language Processing in Python”
Matthew Honnibal, Ines Montani, Sofie Van and Adriane Boyd · 2020
Closest in time.
“Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning”
Eun Jo and Timnit Gebru · 2020
Closest in time.
“Data and its (dis)contents: A survey of dataset development and use in machine learning research”
Amandalynne Paullada et al · 2020
Closest in time.
“Large image datasets: A pyrrhic win for computer vision?”
Original
Vinay Prabhu and Abeba Birhane · 2020
Closest in time.
“How We’ve Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis”
Morgan Scheuerman, Kandrea Wade, Caitlin Lustig and Jed. Brubaker · 2020
Closest in time.
“Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation”
Zeyu Wang et al · 2020
Closest in time.
“Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy”
Kaiyu Yang et al · 2020
Closest in time.
“BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning”
Fisher Yu et al · 2020
Closest in time.
“Amazon SageMaker Clarify”, 2021
Amazon · 2021
Closest in time.
“Algorithmic injustice: a relational ethics approach”
Abeba Birhane · 2021
Closest in time.
“Fairness Flow”, 2021
Facebook AI · 2021
Closest in time.
“Know Your Data”, 2021
Google People + AI Research · 2021
Closest in time.
“Measurement and Fairness”
Abigail. Jacobs and Hanna Wallach · 2021
Closest in time.
“RADIATE: A Radar Dataset for Automotive Perception in Bad Weather”
Marcel Sheeny et al · 2021
Closest in time.
“A Study of Face Obfuscation in ImageNet”
Original
Kaiyu Yang et al · 2021
Closest in time.
“Understanding and Evaluating Racial Biases in Image Captioning”
Original
Dora Zhao, Angelina Wang and Olga Russakovsky · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…