Fetching the paper…
Reading the bibliography…
In the current landscape of language model research, larger models, larger datasets and more compute seems to be the only way to advance towards intelligence.
On measuring social biases in sentence encoders
May, C.; Wang, A.; Bordia, S.; Bowman, S. R.; and Rudinger, R. 2019 · 1903
Earlier work this paper cites.
Measuring bias in contextualized word representations
Kurita, K.; Vyas, N.; Pareek, A.; Black, A. W.; and Tsvetkov, Y. 2019 · 1906
Earlier work this paper cites.
Roberta: A robustly optimized bert pretraining approach
Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019 · 1907
Earlier work this paper cites.
Well-read students learn better: On the importance of pre-training compact models
Turc, I.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019 · 1908
Earlier work this paper cites.
Unsupervised cross-lingual representation learning at scale
Conneau, A.; Khandelwal, K.; Goyal, N.; Chaudhary, V.; Wenzek, G.; Guzmán, F.; Grave, E.; Ott, M.; Zettlemoyer, L.; and Stoyanov, V. 2019 · 1911
Earlier work this paper cites.
CCNet: Extracting high quality monolingual datasets from web crawl data
Wenzek, G.; Lachaux, M.-A.; Conneau, A.; Chaudhary, V.; Guzmán, F.; Joulin, A.; and Grave, E. 2019 · 1911
Earlier work this paper cites.
Scaling laws for neural language models
Kaplan, J.; McCandlish, S.; Henighan, T.; Brown, T. B.; Chess, B.; Child, R.; Gray, S.; Radford, A.; Wu, J.; and Amodei, D. 2020 · 2001
Earlier work this paper cites.
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M.; Bethke, A.; and Reddy, S. 2020 · 2004
Earlier work this paper cites.
Social biases in NLP models as barriers for persons with disabilities
Hutchinson, B.; Prabhakaran, V.; Denton, E.; Webster, K.; Zhong, Y.; and Denuyl, S. 2020 · 2005
Earlier work this paper cites.
Fightin’words: Lexical feature selection and evaluation for identifying the content of political conflict
Monroe, B. L.; Colaresi, M. P.; and Quinn, K. M. 2008 · 2008
Earlier work this paper cites.
Realtoxicityprompts: Evaluating neural toxic degeneration in language models
Gehman, S.; Gururangan, S.; Sap, M.; Choi, Y.; and Smith, N. A. 2020 · 2009
Earlier work this paper cites.
Scaling laws for autoregressive generative modeling
Henighan, T.; Kaplan, J.; Katz, M.; Chen, M.; Hesse, C.; Jackson, J.; Jun, H.; Brown, T. B.; Dhariwal, P.; Gray, S.; et al. 2020 · 2010
Earlier work this paper cites.
UNQOVERing stereotyping biases via underspecified questions
Li, T.; Khot, T.; Khashabi, D.; Sabharwal, A.; and Srikumar, V. 2020 · 2010
Earlier work this paper cites.
CrowS-pairs: A challenge dataset for measuring social biases in masked language models
Nangia, N.; Vania, C.; Bhalerao, R.; and Bowman, S. R. 2020 · 2010
Earlier work this paper cites.
Fairness through awareness
Dwork, C.; Hardt, M.; Pitassi, T.; Reingold, O.; and Zemel, R. 2012 · 2012
Earlier work this paper cites.
Automated experiments on ad privacy settings: A tale of opacity, choice, and discrimination
Datta, A.; Tschantz, M. C.; and Datta, A. 2014 · 2014
Earlier work this paper cites.
Demographic dialectal variation in social media: A case study of African-American English
Blodgett, S. L.; Green, L.; and O’Connor, B. 2016 · 2016
Earlier work this paper cites.
The problem with bias: Allocative versus representational harms in machine learning
Barocas, S.; Crawford, K.; Shapiro, A.; and Wallach, H. 2017 · 2017
Earlier work this paper cites.
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A.; Bryson, J. J.; and Narayanan, A. 2017 · 2017
Earlier work this paper cites.
Deep learning scaling is predictable, empirically
Hestness, J.; Narang, S.; Ardalani, N.; Diamos, G.; Jun, H.; Kianinejad, H.; Patwary, M. M. A.; Yang, Y.; and Zhou, Y. 2017 · 2017
Earlier work this paper cites.
Twitter universal dependency parsing for African-American and mainstream American English
Blodgett, S. L.; Wei, J.; and O’Connor, B. 2018 · 2018
Earlier work this paper cites.
Gender shades: Intersectional accuracy disparities in commercial gender classification
Buolamwini, J.; and Gebru, T. 2018 · 2018
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018 · 2018
Cited alongside, same era.
Measuring and mitigating unintended bias in text classification
Dixon, L.; Li, J.; Sorensen, J.; Thain, N.; and Vasserman, L. 2018 · 2018
Cited alongside, same era.
Gender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J.; Wang, T.; Yatskar, M.; Ordonez, V.; and Chang, K.-W. 2018 · 2018
Cited alongside, same era.
Discrimination through optimization: How Facebook’s Ad delivery can lead to biased outcomes
Ali, M.; Sapiezynski, P.; Bogen, M.; Korolova, A.; Mislove, A.; and Rieke, A. 2019 · 2019
Cited alongside, same era.
What’s in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus
Luccioni, A. S.; and Viviano, J. D. 2021 · 2021
Later among the works it cites.
Auditing algorithms: Understanding algorithmic systems from the outside in
Metaxa, D.; Park, J. S.; Robertson, R. E.; Karahalios, K.; Wilson, C.; Hancock, J.; Sandvig, C.; et al. 2021 · 2021
Later among the works it cites.
BBQ: A hand-built bias benchmark for question answering
Parrish, A.; Chen, A.; Nangia, N.; Padmakumar, V.; Phang, J.; Thompson, J.; Htut, P. M.; and Bowman, S. R. 2021 · 2021
Later among the works it cites.
An introduction to algorithmic fairness
Weerts, H. J. 2021 · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M.; Romanov, A.; Wallach, H.; Chayes, J.; Borgs, C.; Chouldechova, A.; Geyik, S.; Kenthapadi, K.; and Kalai, A. T. 2019 · 2019
Cited alongside, same era.
Women’s syntactic resilience and men’s grammatical luck: Gender-bias in part-of-speech tagging and dependency parsing
Garimella, A.; Banea, C.; Hovy, D.; and Mihalcea, R. 2019 · 2019
Cited alongside, same era.
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
McCoy, T.; Pavlick, E.; and Linzen, T. 2019 · 2019
Cited alongside, same era.
The Risk of Racial Bias in Hate Speech Detection
Sap, M.; Card, D.; Gabriel, S.; Choi, Y.; and Smith, N. A. 2019 · 2019
Cited alongside, same era.
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification
Barbieri, F.; Camacho-Collados, J.; Anke, L. E.; and Neves, L. 2020 · 2020
Cited alongside, same era.
Unmasking Contextual Stereotypes: Measuring and Mitigating BERT’s Gender Bias
Bartl, M.; Nissim, M.; and Gatt, A. 2020 · 2020
Cited alongside, same era.
The pile: An 800gb dataset of diverse text for language modeling
Gao, L.; Biderman, S.; Black, S.; Golding, L.; Hoppe, T.; Foster, C.; Phang, J.; He, H.; Thite, A.; Nabeshima, N.; et al. 2020 · 2020
Cited alongside, same era.
Wei, J.; Bosma, M.; Zhao, V. Y.; Guu, K.; Yu, A. W.; Lester, B.; Du, N.; Dai, A. M.; and Le, Q. V. 2021 · 2021
Later among the works it cites.
Machine bias
Angwin, J.; Larson, J.; Mattu, S.; and Kirchner, L. 2022 · 2022
Later among the works it cites.
Unified scaling laws for routed language models
Clark, A.; De Las Casas, D.; Guy, A.; Mensch, A.; Paganini, M.; Hoffmann, J.; Damoc, B.; Hechtman, B.; Cai, T.; Borgeaud, S.; et al. 2022 · 2022
Later among the works it cites.
Predictability and surprise in large generative models
Ganguli, D.; Hernandez, D.; Lovitt, L.; Askell, A.; Bai, Y.; Chen, A.; Conerly, T.; Dassarma, N.; Drain, D.; Elhage, N.; et al. 2022 · 2022
Later among the works it cites.
Training compute-optimal large language models
Hoffmann, J.; Borgeaud, S.; Mensch, A.; Buchatskaya, E.; Cai, T.; Rutherford, E.; Casas, D. d. L.; Hendricks, L. A.; Welbl, J.; Clark, A.; et al. 2022 · 2022
Later among the works it cites.
Don’t Just Clean It, Proxy Clean It: Mitigating Bias by Proxy in Pre-Trained Models
Panda, S.; Kobren, A.; Wick, M.; and Shen, Q. 2022 · 2022
Later among the works it cites.
Beyond the imitation game: Quantifying and extrapolating the capabilities of language models
Srivastava, A.; Rastogi, A.; Rao, A.; Shoeb, A. A. M.; Abid, A.; Fisch, A.; Brown, A. R.; Santoro, A.; Gupta, A.; Garriga-Alonso, A.; et al. 2022 · 2022
Later among the works it cites.
Upstream mitigation is not all you need: Testing the bias transfer hypothesis in pre-trained language models
Steed, R.; Panda, S.; Kobren, A.; and Wick, M. 2022 · 2022
Later among the works it cites.
Scaling laws vs model architectures: How does inductive bias influence scaling?
Tay, Y.; Dehghani, M.; Abnar, S.; Chung, H. W.; Fedus, W.; Rao, J.; Narang, S.; Tran, V. Q.; Yogatama, D.; and Metzler, D. 2022 · 2022
Later among the works it cites.
Emergent abilities of large language models
Wei, J.; Tay, Y.; Bommasani, R.; Raffel, C.; Zoph, B.; Borgeaud, S.; Yogatama, D.; Bosma, M.; Zhou, D.; Metzler, D.; et al. 2022 · 2022
Later among the works it cites.
Keeping Up with the Language Models: Robustness-Bias Interplay in NLI Data and Models
Baldini, I.; Yadav, C.; Das, P.; and Varshney, K. R. 2023 · 2023
Later among the works it cites.
Fairness and machine learning: Limitations and opportunities
Barocas, S.; Hardt, M.; and Narayanan, A. 2023 · 2023
Later among the works it cites.
On hate scaling laws for data-swamps
Birhane, A.; Prabhu, V.; Han, S.; and Boddeti, V. N. 2023 · 2023
Later among the works it cites.
This prompt is measuring¡ mask¿: evaluating bias evaluation in language models
Goldfarb-Tarrant, S.; Ungless, E.; Balkir, E.; and Blodgett, S. L. 2023 · 2023
Later among the works it cites.
LLM - Detect AI Generated Text
King, J.; Baffour, P.; Crossley, S.; Holbrook, R.; and Demkin, M. 2023 · 2023
Later among the works it cites.
Ms. Categorized: Gender, notability, and inequality on Wikipedia
Tripodi, F. 2023 · 2023
Later among the works it cites.
The Learning Agency Lab - PII Data Detection
Holmes, L.; Crossley, S.; Baffour, P.; King, J.; Burleigh, L.; Demkin, M.; Holbrook, R.; Reade, W.; and Howard, A. 2024 · 2024
Closest in time.