2024

Classist Tools: Social Class Correlates with Performance in NLP

Curry, Amanda Cercas, Attanasio, Giuseppe, Talat, Zeerak et al.

Understand

Since the foundational work of William Labov on the social stratification of language (Labov, 1964), linguistics has made concentrated efforts to explore the links between sociodemographic characteristics and language production and perception.

But while there is strong evidence for socio-demographic characteristics in language, they are infrequently used in Natural Language Processing (NLP).
Age and gender are somewhat well represented, but Labov's original target, socioeconomic status, is noticeably absent.
And yet it matters.

Built on

A survey of race, racism, and anti-racism in NLP
Anjalie Field, Su Lin Blodgett, Zeerak Waseem, and Yulia Tsvetkov. 2021 · 1925
Earlier work this paper cites.
A new readability yardstick
Rudolph Flesch. 1948 · 1948
Earlier work this paper cites.
Language and social class
Basil Bernstein. 1960 · 1960
Earlier work this paper cites.
The social stratification of English in New York city
William Labov. 1964 · 1964
Earlier work this paper cites.
Automated readability index
RJ Senter and Edgar A Smith. 1967 · 1967
Earlier work this paper cites.
The Technique of Clear Writing
Robert Gunning. 1968 · 1968
Earlier work this paper cites.
A computer readability formula designed for machine scoring
Meri Coleman and Ta Lin Liau. 1975 · 1975
Earlier work this paper cites.
Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel
J Peter Kincaid, Robert P Fishburne Jr, Richard L Rogers, and Brad S Chissom. 1975 · 1975
Earlier work this paper cites.
Lix and rix: Variations on a little-known readability index
Jonathan Anderson. 1983 · 1983
Earlier work this paper cites.
The need for new approaches to social class analysis in sociolinguistics
John R Rickford. 1986 · 1986
Earlier work this paper cites.
An initial investigation of the usability of fictional conversation for doing conversation analysis
Alec W McHoul. 1987 · 1987
Earlier work this paper cites.
La sociolingüística de basil bernstein y sus implicaciones en el ámbito escolar
Elisa Usategui Basozábal et al. 1992 · 1992
Earlier work this paper cites.
From WER and RIL to MER and WIL: Improved evaluation measures for connected speech recognition
Andrew Cameron Morris, Viktoria Maier, and Phil Green. 2004 · 2004
Earlier work this paper cites.

Similar

Identity and interaction: A sociocultural linguistic approach
Mary Bucholtz and Kira Hall. 2005 · 2005
Cited alongside, same era.
Television dialogue and natural conversation
Paulo Quaglio. 2008 · 2008
Cited alongside, same era.
Three waves of variation study: The emergence of meaning in the study of sociolinguistic variation
Penelope Eckert. 2012 · 2012
Cited alongside, same era.
A new model of social class? findings from the bbc’s great british class survey experiment
Mike Savage, Fiona Devine, Niall Cunningham, Mark Taylor, Yaojun Li, Johs Hjellbrekke, Brigitte Le Roux, Sam Friedman, and Andrew Miles. 2013 · 2013
Cited alongside, same era.
Socioeconomic status and mortality
Sharon H. Saydah, Giuseppina Imperatore, and Gloria L. Beckles. 2013 · 2013
Cited alongside, same era.
A literature review on the mediation of sociolinguistic style in television and cinematic fiction: Sustaining the ideology of authenticity
Anastasia G Stamou. 2014 · 2014
Cited alongside, same era.
First language acquisition
Eve V Clark and Marisa Casillas. 2015 · 2015
Cited alongside, same era.
Cross-lingual syntactic variation over age and gender
Anders Johannsen, Dirk Hovy, and Anders Søgaard. 2015 · 2015
Cited alongside, same era.
Exploring stylistic variation with age and income on Twitter
Lucie Flekova, Daniel Preoţiuc-Pietro, and Lyle Ungar. 2016 · 2016
Cited alongside, same era.
“How my hair look?” Linguistic authenticity and racialized gender and sexuality on The Wire
Qiuana Lopez and Mary Bucholtz. 2017 · 2017
Cited alongside, same era.
JFLEG: A fluency corpus and benchmark for grammatical error correction
Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. 2017 · 2017
Cited alongside, same era.
Data statements for natural language processing: Toward mitigating system bias and enabling better science
Emily M. Bender and Batya Friedman. 2018 · 2018
Cited alongside, same era.

Then

Gender identity is indexed and perceived in speech
Melanie Weirich and Adrian P Simpson. 2018 · 2018
Later among the works it cites.
Sentence-BERT: Sentence embeddings using Siamese BERT-networks
Nils Reimers and Iryna Gurevych. 2019 · 2019
Later among the works it cites.
wav2vec 2.0: A framework for self-supervised learning of speech representations
Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020 · 2020
Later among the works it cites.
Stanza: A python natural language processing toolkit for many human languages
Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020 · 2020
Later among the works it cites.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020 · 2020
Later among the works it cites.
XLS-R: Self-supervised cross-lingual speech representation learning at scale
Original
Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, et al. 2021 · 2021
Later among the works it cites.
Are AI systems biased against the poor? A machine learning analysis using Word2Vec and GloVe embeddings
Georgina Curto, Mario Fernando Jojoa Acosta, Flavio Comim, and Begoña Garcia-Zapirain. 2022 · 2022
Later among the works it cites.
Demystifying prompts in language models via perplexity estimation
Hila Gonen, Srini Iyer, Terra Blevins, Noah Smith, and Luke Zettlemoyer. 2023 · 2023
Later among the works it cites.
Mistral 7b
Original
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023 · 2023
Later among the works it cites.
Robust speech recognition via large-scale weak supervision
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023 · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models
Original
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023
Later among the works it cites.
Zephyr: Direct distillation of lm alignment
Original
Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, et al. 2023 · 2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…