2017

Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English

Blodgett, Su Lin, O'Connor, Brendan

Understand

We highlight an important frontier in algorithmic fairness: disparity in the quality of natural language processing algorithms when applied to language from authors of different social groups.

For example, current systems sometimes analyze the language of females and minorities more poorly than they do of whites and males.
We conduct an empirical analysis of racial disparity in language identification for tweets written in African-American English, and discuss implications of disparity in NLP.

Built on

Language in the inner city: Studies in the Black English vernacular
William Labov. 1972 · 1972
Earlier work this paper cites.
African American vernacular English: Features, evolution, educational implications
John Russell Rickford. 1999 · 1999
Earlier work this paper cites.
African American English: A Linguistic Introduction
Lisa J. Green. 2002 · 2002
Earlier work this paper cites.
Reconsidering Language Identification for Written Language Resources. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)
Baden Hughes, Timothy Baldwin, Steven Bird, Jeremy Nicholson, and Andrew MacKinlay. 2006 · 2006
Earlier work this paper cites.
Language identification: The long and the short of the matter. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Timothy Baldwin and Marco Lui. 2010 · 2010
Earlier work this paper cites.
langid. py: An Off-the-shelf Language Identification Tool. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL 2012), Demo Session, Jeju, Republic of Korea
M. Lui and T. Baldwin. 2012 · 2012
Earlier work this paper cites.
How Noisy Social Media Text, How Diffrnt Social Media Sources?. In International Joint Conference on Natural Language Processing
Timothy Baldwin, Paul Cook, Marco Lui, Andrew MacKinlay, and Li Wang. 2013 · 2013
Earlier work this paper cites.

Similar

Is the Sample Good Enough? Comparing Data from Twitter’s Streaming API with Twitter’s Firehose. In International AAAI Conference on Weblogs and Social Media
Fred Morstatter, Jürgen Pfeffer, Huan Liu, and Kathleen Carley. 2013 · 2013
Cited alongside, same era.
Discriminative pronunciation modeling for dialectal speech recognition. In Proc. Interspeech
Maider Lehr, Kyle Gorman, and Izhak Shafran. 2014 · 2014
Cited alongside, same era.
Now We Stronger than Ever: African-American English Syntax in Twitter. In Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics
Ian Stewart. 2014 · 2014
Cited alongside, same era.
Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015 · 2015
Cited alongside, same era.
Toward a Description of African American Vernacular English Dialect Regions Using “Black Twitter”
Taylor Jones. 2015 · 2015
Cited alongside, same era.
Challenges of studying and processing dialects in social media. In Proceedings of the Workshop on Noisy User-generated Text
Anna Katrine Jørgensen, Dirk Hovy, and Anders Søgaard. 2015 · 2015
Cited alongside, same era.
Fairness in Learning: Classic and Contextual Bandits
Matthew Joseph, Michael Kearns, Jamie H Morgenstern, and Aaron Roth. 2016a
Cited in the paper.

Then

Demographic Dialectal Variation in Social Media: A Case Study of African-American English
Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016 · 2016
Later among the works it cites.
The Social Impact of Natural Language Processing. In Proceedings of ACL
Dirk Hovy and L. Shannon Spruit. 2016 · 2016
Later among the works it cites.
Learning a POS tagger for AAVE-like language. In Proceedings of NAACL
Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2016 · 2016
Later among the works it cites.
Rawlsian fairness for machine learning
Original
Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, and Aaron Roth. 2016b · 2016
Later among the works it cites.
Combatting police discrimination in the age of big data
Sharad Goel, Maya Perelman, Ravi Shroff, and David Alan Sklansky. 2017 · 2017
Closest in time.
Gender and Dialect Bias in YouTube’s Automatic Captions. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing
Rachael Tatman. 2017 · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…