2023

Toxicity in ChatGPT: Analyzing Persona-assigned Language Models

Deshpande, Ameet, Murahari, Vishvak, Rajpurohit, Tanmay et al.

Understand

Large language models (LLMs) have shown incredible capabilities and transcended the natural language processing (NLP) community, with adoption throughout many services like healthcare, therapy, education, and customer service.

Since users include people with critical information needs like students or patients engaging with chatbots, the safety of these systems is of prime importance.
Therefore, a clear understanding of the capabilities and limitations of LLMs is necessary.
To this end, we systematically evaluate toxicity in over half a million generations of ChatGPT, a popular dialogue-based LLM.

Built on

The probable error of a mean
Student. 1908 · 1908
Earlier work this paper cites.
Scaling laws for neural language models
Original
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020 · 2001
Earlier work this paper cites.
Language models are few-shot learners
Original
Tom B Brown, Benjamin Mann, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2020 · 2005
Earlier work this paper cites.
Misogyny on twitter
Jamie Bartlett, Richard Norrie, Sofia Patel, Rebekka Rumpel, and Simon Wibberley. 2014 · 2014
Earlier work this paper cites.
‘back to the kitchen, cunt’: Speaking the unspeakable about online misogyny
Emma Alice Jane. 2014 · 2014
Earlier work this paper cites.
Character-level convolutional networks for text classification
Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015 · 2015
Earlier work this paper cites.
Semantics derived automatically from language corpora contain human-like biases
Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017 · 2017
Earlier work this paper cites.
Detecting hate speech in social media
Original
Shervin Malmasi and Marcos Zampieri. 2017 · 2017
Earlier work this paper cites.
Men also like shopping: Reducing gender bias amplification using corpus-level constraints
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017 · 2017
Earlier work this paper cites.
Multiple forms of discrimination and relationships with health and wellbeing: findings from national cross-sectional surveys in aotearoa/new zealand
Donna Cormack, James Stanley, and Ricci Harris. 2018 · 2018
Earlier work this paper cites.
Large scale crowdsourcing and characterization of twitter abusive behavior
Antigoni Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018 · 2018
Earlier work this paper cites.
Word embeddings quantify 100 years of gender and ethnic stereotypes
Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018 · 2018
Earlier work this paper cites.
Mitigating unwanted biases with adversarial learning
Original
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018 · 2018
Earlier work this paper cites.
Learning gender-neutral word embeddings
Original
Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018 · 2018
Earlier work this paper cites.
Evaluating the underlying gender bias in contextualized word embeddings
Christine Basta, Marta R. Costa-jussà, and Noe Casas. 2019 · 2019
Earlier work this paper cites.

Similar

Does object recognition work for everyone?
Terrance De Vries, Ishan Misra, Changhan Wang, and Laurens Van der Maaten. 2019 · 2019
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
Measuring bias in contextualized word representations
Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019 · 2019
Cited alongside, same era.
On measuring social biases in sentence encoders
Original
Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019 · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Cited alongside, same era.
The risk of racial bias in hate speech detection
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019 · 2019
Cited alongside, same era.
The woman worked as a babysitter: On biases in language generation
Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019 · 2019
Cited alongside, same era.
Universal adversarial triggers for attacking and analyzing nlp
Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019 · 2019
Cited alongside, same era.
Gender bias in contextualized word embeddings
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019 · 2019
Cited alongside, same era.
Language (technology) is power: A critical survey of “bias” in NLP
Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020 · 2020
Cited alongside, same era.
Queens are powerful too: Mitigating gender bias in dialogue generation
Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela, and Jason Weston. 2020 · 2020
Cited alongside, same era.
Realtoxicityprompts: Evaluating neural toxic degeneration in language models
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020 · 2020
Cited alongside, same era.

Then

Probing toxic content in large pre-trained language models
Nedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song, and Dit-Yan Yeung. 2021 · 2021
Later among the works it cites.
Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in nlp
Timo Schick, Sahana Udupa, and Hinrich Schütze. 2021 · 2021
Later among the works it cites.
Universal adversarial attacks with natural triggers for text classification
Liwei Song, Xinwei Yu, Hsuan-Tung Peng, and Karthik Narasimhan. 2021 · 2021
Later among the works it cites.
Racism in europe: Characteristics and intersections with other social categories
Elena Ball, Melanie C Steffens, and Claudia Niedlich. 2022 · 2022
Later among the works it cites.
Palm: Scaling language modeling with pathways
Original
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022 · 2022
Later among the works it cites.
Mitigating toxic degeneration with empathetic data: Exploring the relationship between toxicity and empathy
Allison Lahnala, Charles Welch, Béla Neuendorf, and Lucie Flek. 2022 · 2022
Later among the works it cites.
Holistic evaluation of language models
Original
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. 2022 · 2022
Later among the works it cites.
Training language models to follow instructions with human feedback
Original
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022
Later among the works it cites.
Leashing the inner demons: Self-detoxification for language models
Canwen Xu, Zexue He, Zhankui He, and Julian McAuley. 2022 · 2022
Later among the works it cites.
A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity
Original
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023 · 2023
Closest in time.
Reward modeling for mitigating toxicity in transformer-based language models
Farshid Faal, Ketra Schmitt, and Jia Yuan Yu. 2023 · 2023
Closest in time.
Introducing chatgpt
OpenAI. 2023 · 2023
Closest in time.
Perspective api
Perspective. 2023 · 2023
Closest in time.
Exploring ai ethics of chatgpt: A diagnostic analysis
Original
Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. 2023 · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…