2022

Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation

Goyal, Nitesh, Kivlichan, Ian, Rosen, Rachel et al.

Understand

Machine learning models are commonly used to detect toxicity in online conversations.

These models are trained on datasets annotated by human raters.
We explore how raters' self-described identities impact how they annotate toxicity in online comments.
We first define the concept of specialized rater pools: rater pools formed based on raters' self-described identities, rather than at random.

Built on

Logistic regression diagnostics
Daryl Pregibon. 1981 · 1981
Earlier work this paper cites.
Basic content analysis (2 ed.)
Robert Philip Weber. 1990 · 1990
Earlier work this paper cites.
Categorical Data Analysis . Wiley Series in Probability and Statistics, Vol. 482
Alan Agresti. 2002 · 2002
Earlier work this paper cites.
Income security in the on-demand economy: Findings and policy lessons from a survey of crowdworkers
Janine Berg. 2015 · 2015
Earlier work this paper cites.
Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter. In Proceedings of the First Workshop on NLP and Computational Social Science . Association for Computational Linguistics, Austin, Texas, 138–142
Zeerak Waseem. 2016 · 2016
Earlier work this paper cites.
Like Trainer, Like Bot? Inheritance of Bias in Algorithmic Content Moderation. In Social Informatics , Giovanni Luca Ciampaglia, Afra Mashhadi, and Taha Yasseri (Eds.). Springer International Publishing, Cham, 405–415
Reuben Binns, Michael Veale, Max Van Kleek, and Nigel Shadbolt. 2017 · 2017
Earlier work this paper cites.
Ex Machina: Personal Attacks Seen at Scale. In Proceedings of the 26th International Conference on World Wide Web (Perth, Australia) (WWW ’17) . International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1391–1399
Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017 · 2017
Earlier work this paper cites.
Measuring and Mitigating Unintended Bias in Text Classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (New Orleans, LA, USA) (AIES ’18) . Association for Computing Machinery, New York, NY, USA, 67–73
Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018 · 2018
Earlier work this paper cites.
In US, estimate of LGBT population rises to 4.5%
Frank Newport. 2018 · 2018
Earlier work this paper cites.
Characterizing the global crowd workforce: A cross-country comparison of crowdworker demographics
Original
Lisa Posch, Arnim Bleier, Fabian Flöck, and Markus Strohmaier. 2018 · 2018
Earlier work this paper cites.
Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification. In Companion Proceedings of The 2019 World Wide Web Conference (San Francisco, USA) (WWW ’19) . Association for Computing Machinery, New York, NY, USA, 491–500
Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019 · 2019
Earlier work this paper cites.
Racial Bias in Hate Speech and Abusive Language Detection Datasets. In Proceedings of the Third Workshop on Abusive Language Online . Association for Computational Linguistics, Florence, Italy, 25–35
Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019 · 2019
Earlier work this paper cites.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) . Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Earlier work this paper cites.
Black vs. African-American: The complex conversation Black Americans are having about identity #ForTheCulture
Michael Quander and Lauryn Froneberger. 2019 · 2019
Earlier work this paper cites.
The Risk of Racial Bias in Hate Speech Detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Association for Computational Linguistics, Florence, Italy, 1668–1678
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019 · 2019
Earlier work this paper cites.

Similar

Modeling Annotator Perspective and Polarized Opinions to Improve Hate Speech Detection
Sohail Akhtar, Valerio Basile, and Viviana Patti. 2020 · 2020
Cited alongside, same era.
Identifying and Measuring Annotator Bias Based on Annotators’ Demographic Characteristics. In Proceedings of the Fourth Workshop on Online Abuse and Harms . Association for Computational Linguistics, Online, 184–190
Hala Al Kuwatly, Maximilian Wich, and Georg Groh. 2020 · 2020
Cited alongside, same era.
Deep learning models for multilingual hate speech detection
Original
Sai Saketh Aluru, Binny Mathew, Punyajoy Saha, and Animesh Mukherjee. 2020 · 2020
Cited alongside, same era.
It’s the End of the Gold Standard as We Know It. In AIxIA 2020 – Advances in Artificial Intelligence , Matteo Baldoni and Stefania Bandini (Eds.). Springer International Publishing, Cham, 441–453
Valerio Basile. 2021 · 2020
Cited alongside, same era.
Annotation instructions for Toxicity with sub-attributes
Lucas Dixon. 2018 · 2020
Cited alongside, same era.
Cyberbullying Detection With Fairness Constraints
Oguzhan Gencoglu. 2021 · 2020
Cited alongside, same era.
Drag queens and Artificial Intelligence: should computers decide what is ‘toxic’ on the internet?
Alessandra Gomes, Dennys Antonialli, and Thiago Dias Oliva. 2019 · 2020
Cited alongside, same era.
Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition. In Proceedings of the 12th Language Resources and Evaluation Conference . European Language Resources Association, Marseille, France, 1440–1448
Xiaolei Huang, Linzi Xing, Franck Dernoncourt, and Michael J. Paul. 2020 · 2020
Cited alongside, same era.
Biased Priorities, Biased Outcomes: Three Recommendations for Ethics-Oriented Data Annotation Practices. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (New York, NY, USA) (AIES ’20) . Association for Computing Machinery, New York, NY, USA, 71
Gunay Kazimzade and Milagros Miceli. 2020 · 2020
Cited alongside, same era.
Towards a Comprehensive Taxonomy and Large-Scale Annotated Corpus for Online Slur Usage. In Proceedings of the Fourth Workshop on Online Abuse and Harms . Association for Computational Linguistics, Online, 138–149
Jana Kurrek, Haji Mohammad Saleem, and Derek Ruths. 2020 · 2020
Cited alongside, same era.
Investigating Annotator Bias with a Graph-Based Approach. In Proceedings of the Fourth Workshop on Online Abuse and Harms . Association for Computational Linguistics, Online, 191–199
Maximilian Wich, Hala Al Kuwatly, and Georg Groh. 2020a · 2020
Cited alongside, same era.
Impact of Politically Biased Data on Hate Speech Classification. In Proceedings of the Fourth Workshop on Online Abuse and Harms . Association for Computational Linguistics, Online, 54–64
Maximilian Wich, Jan Bauer, and Georg Groh. 2020b · 2020
Cited alongside, same era.

Then

Hate versus politics: detection of hate against policy makers in Italian tweets
Armend Duzha, Cristiano Casadei, Michael Tosi, and Fabio Celli. 2021 · 2021
Later among the works it cites.
Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework. In Equity and Access in Algorithms, Mechanisms, and Optimization (New York, NY, USA) (EAAMO ’21) . Association for Computing Machinery, New York, NY, USA, Article 7, 11 pages
Matan Halevy, Camille Harris, Amy Bruckman, Diyi Yang, and Ayanna Howard. 2021 · 2021
Later among the works it cites.
Offensive, aggressive, and hate speech analysis: From data-centric to human-centered approach
Jan Kocoń, Alicja Figas, Marcin Gruza, Daria Puchalska, Tomasz Kajdanowicz, and Przemysław Kazienko. 2021 · 2021
Later among the works it cites.
Reconsidering Annotator Disagreement about Racist Language: Noise or Signal?. In Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media . Association for Computational Linguistics, Online, 81–90
Savannah Larimore, Ian Kennedy, Breon Haskett, and Alina Arseniev-Koehler. 2021 · 2021
Later among the works it cites.
The Authors Matter: Understanding and Mitigating Implicit Bias in Deep Text Classification. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 . Association for Computational Linguistics, Online, 74–85
Haochen Liu, Wei Jin, Hamid Karimi, Zitao Liu, and Jiliang Tang. 2021 · 2021
Later among the works it cites.
Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysis
Original
Saif M Mohammad. 2021 · 2021
Later among the works it cites.
Understanding and Interpreting the Impact of User Context in Hate Speech Detection. In Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media . Association for Computational Linguistics, Online, 91–102
Edoardo Mosca, Maximilian Wich, and Georg Groh. 2021 · 2021
Later among the works it cites.
HateCheck: Functional Tests for Hate Speech Detection Models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Association for Computational Linguistics, Online, 41–58
Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. 2021 · 2021
Later among the works it cites.
Evaluating Debiasing Techniques for Intersectional Biases. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 2492–2498
Shivashankar Subramanian, Xudong Han, Timothy Baldwin, Trevor Cohn, and Lea Frermann. 2021 · 2021
Later among the works it cites.
QuickFacts: United States
U.S. Census Bureau. 2019 · 2021
Later among the works it cites.
Assessing Multilingual Fairness in Pre-trained Multimodal Representations
Original
Jialu Wang, Yang Liu, and Xin Eric Wang. 2021 · 2021
Later among the works it cites.
Introducing an Abusive Language Classification Framework for Telegram to Investigate the German Hater Community
Original
Maximilian Wich, Adrian Gorniak, Tobias Eder, Daniel Bartmann, Burak Enes Cakici, and Georg Groh. 2021 · 2021
Later among the works it cites.
Computational Models of Identity Presentation in Language
Michael Miller Yoder. 2021 · 2021
Later among the works it cites.
Annotating Online Misogyny. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Association for Computational Linguistics, Online, 3181–3197
Philine Zeinert, Nanna Inie, and Leon Derczynski. 2021 · 2021
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…