Fetching the paper…
Reading the bibliography…
Recent years have seen the development of many open-source ML fairness toolkits aimed at helping ML practitioners assess and address unfairness in their systems.
Thematic analysis
Victoria Clarke and Virginia Braun. 2014 · 1952
Earlier work this paper cites.
The think aloud method: a practical approach to modelling cognitive processes
Maarten van Someren, Yvonne Barnard, and Jacobijn A. C. Sandberg. 1994 · 1994
Earlier work this paper cites.
Python reference manual
Guido Van Rossum and Fred L Drake Jr. 1995 · 1995
Earlier work this paper cites.
Becoming Wikipedian: transformation of participation in a collaborative online encyclopedia. In GROUP ’05
Sue Lacey Bryant, Andrea Forte, and Amy Bruckman. 2005 · 2005
Earlier work this paper cites.
Getting teams to talk: development and pilot implementation of a checklist to promote interprofessional communication in the OR
Lorelei A Lingard, Sherry Espin, Barbara Rubin, Sarah Whyte, Marcela Colmenares, G. Ross Baker, Diane Doran, Ethan D. Grober, Beverley A. Orser, John Bohnen, and Richard Reznick. 2005 · 2005
Earlier work this paper cites.
Using thematic analysis in psychology
Virginia Braun and Victoria Clarke. 2006 · 2006
Earlier work this paper cites.
Design Guidance for Emergency and Abnormal Checklists in Aviation
Barbara Katrina Burian. 2006 · 2006
Earlier work this paper cites.
The checklist–a tool for error management and performance improvement
Brigette M. Hales and Peter J. Pronovost. 2006 · 2006
Earlier work this paper cites.
The factory pattern in API design: A usability evaluation. In 29th International Conference on Software Engineering (ICSE’07) . IEEE, 302–312
Brian Ellis, Jeffrey Stylos, and Brad Myers. 2007 · 2007
Earlier work this paper cites.
Using data mining to predict secondary school student performance
Paulo Cortez and Alice Maria Gonçalves Silva. 2008 · 2008
Earlier work this paper cites.
The implications of method placement on API learnability. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering . 105–112
Jeffrey Stylos and Brad A Myers. 2008 · 2008
Earlier work this paper cites.
Modern Construction : Lean Project Delivery and Integrated Practices
Lincoln H. Forbes and Syed M. Ahmed. 2010 · 2010
Earlier work this paper cites.
Comment: Snowball versus Respondent-Driven Sampling
Douglas D. Heckathorn. 2011 · 2011
Earlier work this paper cites.
The state of the art in end-user software engineering
Andrew J Ko, Robin Abraham, Laura Beckwith, Alan Blackwell, Margaret Burnett, Martin Erwig, Chris Scaffidi, Joseph Lawrance, Henry Lieberman, Brad Myers, et al · 2011
Earlier work this paper cites.
pandas: a foundational Python library for data analysis and statistics
Wes McKinney et al · 2011
Earlier work this paper cites.
Scikit-learn: Machine Learning in Python
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011 · 2011
Earlier work this paper cites.
Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference . 214–226
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012 · 2012
Earlier work this paper cites.
The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline
Aaron Halfaker, R Stuart Geiger, Jonathan T Morgan, and John Riedl. 2013 · 2013
Earlier work this paper cites.
TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems
Martín Abadi and Ashish Agarwal et al. 2015 · 2015
Earlier work this paper cites.
" I always assumed that I wasn’t really that close to [her]" Reasoning about Invisible Algorithms in News Feeds. In Proceedings of the 33rd annual ACM conference on human factors in computing systems . 153–162
Motahhare Eslami, Aimee Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie Karahalios, Kevin Hamilton, and Christian Sandvig. 2015 · 2015
Earlier work this paper cites.
We are dynamo: Overcoming stalling and friction in collective action for crowd workers. In Proceedings of the 33rd annual ACM conference on human factors in computing systems . 1621–1630
Niloufar Salehi, Lilly C Irani, Michael S Bernstein, Ali Alkhatib, Eva Ogbe, and Kristy Milland. 2015 · 2015
Earlier work this paper cites.
Searching the state space: A qualitative study of API protocol usability. In 2015 IEEE 23rd International Conference on Program Comprehension . IEEE, 82–93
Joshua Sushine, James D Herbsleb, and Jonathan Aldrich. 2015 · 2015
Earlier work this paper cites.
FairML: ToolBox for diagnosing bias in predictive modeling
Julius A Adebayo et al · 2016
Earlier work this paper cites.
Machine bias
Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016 · 2016
Earlier work this paper cites.
Man is to computer programmer as woman is to homemaker? debiasing word embeddings
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016 · 2016
Earlier work this paper cites.
Detecting student emotions in computer-enabled classrooms.. In IJCAI . 4125–4129
Nigel Bosch, Sidney K D’Mello, Ryan S Baker, Jaclyn Ocumpaugh, Valerie Shute, Matthew Ventura, Lubin Wang, and Weinan Zhao. 2016 · 2016
Earlier work this paper cites.
A computer program used for bail and sentencing decisions was labeled biased against blacks. It’s actually not that clear
Avi Feller, Emma Pierson, Sam Corbett-Davies, and Sharad Goel. 2016 · 2016
Earlier work this paper cites.
Equality of opportunity in supervised learning
Moritz Hardt, Eric Price, and Nati Srebro. 2016 · 2016
Earlier work this paper cites.
Programmers Are Users Too: Human-Centered Methods for Improving Programming Tools
Brad A. Myers, Amy J. Ko, Thomas D. LaToza, and YoungSeok Yoon. 2016 · 2016
Earlier work this paper cites.
Improving API usability
Brad A. Myers and Jeffrey Stylos. 2016 · 2016
Earlier work this paper cites.
Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images
Korsuk Sirinukunwattana, Shan e Ahmed Raza, Yee-Wah Tsang, David R. J. Snead, Ian A. Cree, and Nasir M. Rajpoot. 2016 · 2016
Earlier work this paper cites.
Facets - visualizations for ML datasets
2017 · 2017
Earlier work this paper cites.
Fair prediction with disparate impact: A study of bias in recidivism prediction instruments
Alexandra Chouldechova. 2017 · 2017
Earlier work this paper cites.
Dermatologist-level classification of skin cancer with deep neural networks
Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sebastian Thrun. 2017 · 2017
Earlier work this paper cites.
Data vision: Learning to see through algorithmic abstraction. In Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing . 2436–2447
Samir Passi and Steven Jackson. 2017 · 2017
Earlier work this paper cites.
Automatic differentiation in PyTorch
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017 · 2017
Earlier work this paper cites.
R: A Language and Environment for Statistical Computing
R Core Team. 2017 · 2017
Earlier work this paper cites.
FairTest: Discovering unwarranted associations in data-driven applications. In 2017 IEEE European Symposium on Security and Privacy (EuroS&P) . IEEE, 401–416
Florian Tramer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, Jean-Pierre Hubaux, Mathias Humbert, Ari Juels, and Huang Lin. 2017 · 2017
Cited alongside, same era.
Towards algorithmic experience: Initial efforts for social media contexts. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems . 1–12
Oscar Alvarado and Annika Waern. 2018 · 2018
Cited alongside, same era.
Themis-ml: A fairness-aware machine learning interface for end-to-end discrimination discovery and mitigation
Niels Bantilan. 2018 · 2018
Cited alongside, same era.
Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research, Vol. 81) , Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, 77–91
Joy Buolamwini and Timnit Gebru. 2018 · 2018
Cited alongside, same era.
Opportunities for a More Interdisciplinary Approach to Perceptions of Fairness in Machine Learning
Sophia T. Dasch, Vincent Rice, Venkat R. Lakshminarayanan, Taiwo A. Togun, C. Malik Boykin, and Sarah M. Brown. 2020 · 2020
Later among the works it cites.
A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
Maria De-Arteaga, Riccardo Fogliato, and Alexandra Chouldechova. 2020 · 2020
Later among the works it cites.
Shifting Concepts of Value: Designing Algorithmic Decision-Support Systems for Public Services
Naja Holten Møller, Irina Shklovski, and Thomas T. Hildebrandt. 2020 · 2020
Later among the works it cites.
Bias mitigation with AIF360: A comparative study. In Norsk IKT-konferanse for forskning og utdanning
Knut T Hufthammer, Tor H Aasheim, Sølve Ånneland, Håvard Brynjulfsen, and Marija Slavkovik. 2020 · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction. In Proceedings of the 2018 World Wide Web Conference (Lyon, France) (WWW ’18) . International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 903–912
Nina Grgic-Hlaca, Elissa M. Redmiles, Krishna P. Gummadi, and Adrian Weller. 2018 · 2018
Cited alongside, same era.
The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards
Sarah Holland, Ahmed Hosny, Sarah Newman, Joshua Joseph, and Kasia Chmielinski. 2018 · 2018
Cited alongside, same era.
Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In International Conference on Machine Learning . PMLR, 2564–2572
Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2018 · 2018
Cited alongside, same era.
Inherent trade-offs in algorithmic fairness. In Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems . 40–40
Jon Kleinberg. 2018 · 2018
Cited alongside, same era.
Prediction-based decisions and fairness: A catalogue of choices, assumptions, and definitions
Shira Mitchell, Eric Potash, Solon Barocas, Alexander D’Amour, and Kristian Lum. 2018 · 2018
Cited alongside, same era.
API Designers in the Field: Design Practices and Challenges for Creating Usable APIs
Lauren Murphy, Mary Beth Kery, Oluwatosin Alliyu, Andrew Peter Macvean, and Brad A. Myers. 2018 · 2018
Cited alongside, same era.
Trust in data science: Collaboration, translation, and accountability in corporate data science projects
Samir Passi and Steven J Jackson. 2018 · 2018
Cited alongside, same era.
Data lifecycle challenges in production machine learning: a survey
Neoklis Polyzotis, Sudip Roy, Steven Euijong Whang, and Martin Zinkevich. 2018 · 2018
Cited alongside, same era.
Brittany Johnson, Jesse Bartola, Rico Angell, Katherine Keith, Sam Witty, Stephen J Giguere, and Yuriy Brun. 2020 · 2020
Later among the works it cites.
Racial disparities in automated speech recognition
Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R Rickford, Dan Jurafsky, and Sharad Goel. 2020 · 2020
Later among the works it cites.
Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems . 1–14
Michael A Madaio, Luke Stark, Jennifer Wortman Vaughan, and Hanna Wallach. 2020 · 2020
Later among the works it cites.
Between subjectivity and imposition: Power dynamics in data annotation for computer vision
Milagros Miceli, Martin Schuessler, and Tianling Yang. 2020 · 2020
Later among the works it cites.
Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2020 · 2020
Later among the works it cites.
Factors influencing perceived fairness in algorithmic decision-making: Algorithm outcomes, development procedures, and individual differences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems . 1–14
Ruotong Wang, F Maxwell Harper, and Haiyi Zhu. 2020 · 2020
Later among the works it cites.
Keeping Designers in the Loop: Communicating Inherent Algorithmic Trade-offs Across Multiple Objectives
Bowen Yu, Ye Yuan, Loren G. Terveen, Zhiwei Steven Wu, Jodi Forlizzi, and Haiyi Zhu. 2020 · 2020
Later among the works it cites.
How do Data Science Workers Collaborate? Roles, Workflows, and Tools
Amy X. Zhang, Michael Muller, and Dakuo Wang. 2020 · 2020
Later among the works it cites.
People AI Guidebook
2021 · 2021
Later among the works it cites.
What We Can’t Measure, We Can’t Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency . 249–260
McKane Andrus, Elena Spitzer, Jeffrey Brown, and Alice Xiang. 2021 · 2021
Later among the works it cites.
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency . 610–623
Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021 · 2021
Later among the works it cites.
Datasheets for Datasets help ML Engineers Notice and Understand Ethical Issues in Training Data
Karen Boyd. 2021 · 2021
Later among the works it cites.
Discovering and Validating AI Errors With Crowdsourced Failure Reports
Ángel Alexander Cabrera, Abraham J Druck, Jason I Hong, and Adam Perer. 2021 · 2021
Later among the works it cites.
Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (Virtual Event, Canada) (FAccT ’21) . Association for Computing Machinery, New York, NY, USA, 598–609
Jennifer Cobbe, Michelle Seng Ah Lee, and Jatinder Singh. 2021 · 2021
Later among the works it cites.
On the genealogy of machine learning datasets: A critical history of ImageNet
Emily Denton, Alex Hanna, Razvan Amironesei, Andrew Smart, and Hilary Nicole. 2021 · 2021
Later among the works it cites.
Datasheets for datasets
Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. 2021 · 2021
Later among the works it cites.
Soumya Ghosh, Q Vera Liao, Karthikeyan Natesan Ramamurthy, Jiri Navratil, Prasanna Sattigeri, Kush R Varshney, and Yunfeng Zhang. 2021 · 2021
Later among the works it cites.
Ten Million Users and Ten Years Later: Python Tutor’s Design Guidelines for Building Scalable and Sustainable Research Software in Academia. In The 34th Annual ACM Symposium on User Interface Software and Technology . 1235–1251
Philip Guo. 2021 · 2021
Later among the works it cites.
Designing for human-AI complementarity in K-12 education
Kenneth Holstein and Vincent Aleven. 2021 · 2021
Later among the works it cites.
Formalising trade-offs beyond algorithmic fairness: lessons from ethical philosophy and welfare economics
Michelle Seng Ah Lee, Luciano Floridi, and Jatinder Singh. 2021 · 2021
Later among the works it cites.
The landscape and gaps in open source fairness toolkits. In Proceedings of the 2021 CHI conference on human factors in computing systems . 1–13
Michelle Seng Ah Lee and Jat Singh. 2021a · 2021
Later among the works it cites.
Assessing the Fairness of AI Systems: AI Practitioners’ Processes, Challenges, and Needs for Support
Michael Madaio, Lisa Egede, Hariharan Subramonyam, Jennifer Wortman Vaughan, and Hanna Wallach. 2021 · 2021
Later among the works it cites.
Data and its (dis) contents: A survey of dataset development and use in machine learning research
Amandalynne Paullada, Inioluwa Deborah Raji, Emily M Bender, Emily Denton, and Alex Hanna. 2021 · 2021
Later among the works it cites.
Where responsible AI meets reality: Practitioner perspectives on enablers for shifting organizational practices
Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2021 · 2021
Later among the works it cites.
Towards Fairness in Practice: A Practitioner-Oriented Rubric for Evaluating Fair ML Toolkits. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1–13
Brianna Richardson, Jean Garcia-Gathright, Samuel F Way, Jennifer Thom, and Henriette Cramer. 2021 · 2021
Later among the works it cites.
Re-imagining algorithmic fairness in india and beyond. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency . 315–328
Nithya Sambasivan, Erin Arnesen, Ben Hutchinson, Tulsee Doshi, and Vinodkumar Prabhakaran. 2021a · 2021
Later among the works it cites.
Do datasets have politics? Disciplinary values in computer vision dataset development
Morgan Klaus Scheuerman, Alex Hanna, and Emily Denton. 2021 · 2021
Later among the works it cites.
Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberation. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency . 850–861
Hong Shen, Wesley H Deng, Aditi Chattopadhyay, Zhiwei Steven Wu, Xu Wang, and Haiyi Zhu. 2021a · 2021
Later among the works it cites.
Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors
Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021b · 2021
Later among the works it cites.
Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1–13
Niels van Berkel, Jorge Goncalves, Daniel Russo, Simo Hosio, and Mikael B Skov. 2021 · 2021
Later among the works it cites.
How Much Automation Does a Data Scientist Want?
Dakuo Wang, Q Vera Liao, Yunfeng Zhang, Udayan Khurana, Horst Samulowitz, Soya Park, Michael Muller, and Lisa Amini. 2021 · 2021
Later among the works it cites.
Timelines: A World-Building Activity for Values Advocacy. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1–15
Richmond Y Wong and Tonya Nguyen. 2021 · 2021
Later among the works it cites.
Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?
Milagros Miceli, Julian Posada, and Tianling Yang. 2022 · 2022
Closest in time.