2023

How Prevalent is Gender Bias in ChatGPT? -- Exploring German and English ChatGPT Responses

Urchs, Stefanie, Thurner, Veronika, Aßenmacher, Matthias et al.

Understand

With the introduction of ChatGPT, OpenAI made large language models (LLM) accessible to users with limited IT expertise.

However, users with no background in natural language processing (NLP) might lack a proper understanding of LLMs.
Thus the awareness of their inherent limitations, and therefore will take the systems' output at face value.
In this paper, we systematically analyse prompts and the generated responses to identify possible problematic issues with a special focus on gender biases, which users need to be aware of when processing the system's output.

Built on

Lakoff, R.: Language and woman’s place. Language in Society 2
1973
Earlier work this paper cites.
Trix, F., Psenka, C.: Exploring the color of glass: Letters of recommendation for female and male medical faculty. Discourse & Society 14
2003
Earlier work this paper cites.
Gaucher, D., Friesen, J., Kay, A.C.: Evidence that gendered wording in job advertisements exists and sustains gender inequality. Journal of personality and social psychology 101
2011
Earlier work this paper cites.
Kulesza, T., Stumpf, S., Burnett, M., Kwan, I.: Tell me more? the effects of mental model soundness on personalizing an intelligent agent. In: Proceedings of the sigchi conference on human factors in computing systems. p. 1–10. CHI ’12, Association for Computing Machinery, New York, NY, USA (2012). https://doi.org/10.1145/2207676.2207678, https://doi.org/10.1145/2207676.2207678
2012
Earlier work this paper cites.
Graells-Garrido, E., Lalmas, M., Menczer, F.: First women, second sex: Gender bias in wikipedia. In: Proceedings of the 26th ACM Conference on Hypertext and Social Media. p. 165–174. HT ’15, Association for Computing Machinery, New York, NY, USA (2015). https://doi.org/10.1145/2700171.2791036, https://doi.org/10.1145/2700171.2791036
2015
Earlier work this paper cites.
Fu, L., Danescu-Niculescu-Mizil, C., Lee, L.: Tie-breaker: Using language models to quantify gender bias in sports journalism. In: Proceedings of the IJCAI workshop on NLP meets Journalism (2016)
2016
Earlier work this paper cites.
Horvath, L.K., Sczesny, S.: Reducing women’s lack of fit with leadership positions? effects of the wording of job advertisements. European Journal of Work and Organizational Psychology 25
2016
Earlier work this paper cites.
Sczesny, S., Formanowicz, M., Moser, F.: Can gender-fair language reduce gender stereotyping and discrimination? Frontiers in psychology p. 25 (2016)
2016
Earlier work this paper cites.

Similar

Bailey, A.H., LaFrance, M.: Who counts as human? antecedents to androcentric behavior. Sex Roles 76
2017
Cited alongside, same era.
Nakandala, S., Ciampaglia, G., Su, N., Ahn, Y.Y.: Gendered conversation in a social game-streaming platform. Proceedings of the International AAAI Conference on Web and Social Media 11
2017
Cited alongside, same era.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., Polosukhin, I.: Attention is All you Need. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017), https://proceedings.neurips.cc/paper˙files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
2017
Cited alongside, same era.
Madaan, N., Mehta, S., Agrawaal, T., Malhotra, V., Aggarwal, A., Gupta, Y., Saxena, M.: Analyze, detect and remove gender stereotyping from bollywood movies. In: Conference on fairness, accountability and transparency. pp. 92–105. PMLR (2018)
2018
Cited alongside, same era.
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota (2019). https://doi.org/10.18653/v1/N19-1423, https://aclanthology.org/N19-1423
2019
Cited alongside, same era.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. OpenAi Blog (2019)
2019
Cited alongside, same era.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. Advances in neural information processing systems 33
2020
Cited alongside, same era.
D’ignazio, C., Klein, L.F.: Data feminism. MIT press (2020)
2020
Cited alongside, same era.

Then

Dutz, R., Rehbock, S., Peus, C.: Führmint gender decoder: Subtile geschlechtskodierung in stellenanzeigen erkennen und auflösen [führmint gender decoder: Identifying and resolving subtle gender coding in job advertisements]. Personal in Hochschule und Wissenschaft entwickeln (2020). https://doi.org/no DOI available
2020
Later among the works it cites.
Field, A., Tsvetkov, Y.: Unsupervised discovery of implicit gender bias. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 596–608 (2020)
2020
Later among the works it cites.
Mateo, C.M., Williams, D.R.: More than words: a vision to address bias and reduce discrimination in the health professions learning environment. Academic medicine 95
2020
Later among the works it cites.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research 21
2020
Later among the works it cites.
Fast, E., Vachovsky, T., Bernstein, M.: Shirtless and dangerous: Quantifying linguistic signals of gender bias in an online fiction writing community. Proceedings of the International AAAI Conference on Web and Social Media 10
2021
Later among the works it cites.
Wagner, C., Garcia, D., Jadidi, M., Strohmaier, M.: It’s a man’s wikipedia? assessing gender inequality in an online encyclopedia. Proceedings of the International AAAI Conference on Web and Social Media 9
2021
Later among the works it cites.
OpenAI: Chatgpt: Optimizing language models for dialogue (2022), https://openai.com/blog/chatgpt/
2022
Later among the works it cites.
OpenAI: ChatGPT(May 24 version). [Large Language Model] (2023), https://chat.openai.com/chat
2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…