2023

A Categorical Archive of ChatGPT Failures

Borji, Ali

Understand

Large language models have been demonstrated to be valuable in different fields.

ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation by comprehending context and generating appropriate responses.
It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries, with fluent and comprehensive answers surpassing prior public chatbots in both security and usefulness.
However, a comprehensive analysis of ChatGPT's failures is lacking, which is the focus of this study.

Built on

Consciousness and language
John R Searle, S Willis, et al · 2002
Earlier work this paper cites.
A test for consciousness
Christof Koch and Giulio Tononi · 2011
Earlier work this paper cites.
The winograd schema challenge
Hector Levesque, Ernest Davis, and Leora Morgenstern · 2012
Earlier work this paper cites.
Theory of mind: Mechanisms, methods, and new directions
Lindsey J Byom and Bilge Mutlu · 2013
Earlier work this paper cites.
A large annotated corpus for learning natural language inference
Original
Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning · 2015
Earlier work this paper cites.
Gender shades: Intersectional accuracy disparities in commercial gender classification
Joy Buolamwini and Timnit Gebru · 2018
Earlier work this paper cites.
Mathqa: Towards interpretable math word problem solving with operation-based formalisms
Original
Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi · 2019
Earlier work this paper cites.
Pros and cons of gan evaluation measures
Ali Borji · 2019
Earlier work this paper cites.
Megatron-lm: Training multi-billion parameter language models using model parallelism
Original
Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro · 2019
Earlier work this paper cites.
Energy and policy considerations for deep learning in nlp
Original
Emma Strubell, Ananya Ganesh, and Andrew McCallum · 2019
Earlier work this paper cites.
Superglue: A stickier benchmark for general-purpose language understanding systems
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman · 2019
Earlier work this paper cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Earlier work this paper cites.
The carbon impact of artificial intelligence
Payal Dhar · 2020
Earlier work this paper cites.
Stereoset: Measuring stereotypical bias in pretrained language models
Original
Moin Nadeem, Anna Bethke, and Siva Reddy · 2020
Earlier work this paper cites.
Investigating gender bias in language models using causal mediation analysis
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber · 2020
Earlier work this paper cites.
Concealed data poisoning attacks on nlp models
Original
Eric Wallace, Tony Z Zhao, Shi Feng, and Sameer Singh · 2020
Earlier work this paper cites.
Persistent anti-muslim bias in large language models
Abubakar Abid, Maheen Farooqi, and James Zou · 2021
Earlier work this paper cites.
Extracting training data from large language models
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B Brown, Dawn Song, Ulfar Erlingsson, et al · 2021
Earlier work this paper cites.
Evaluating large language models trained on code
Original
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al · 2021
Earlier work this paper cites.
Training verifiers to solve math word problems
Original
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al · 2021
Earlier work this paper cites.
Measuring mathematical problem solving with the math dataset
Original
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt · 2021
Earlier work this paper cites.
Sustainable modular debiasing of language models
Original
Anne Lauscher, Tobias Lueken, and Goran Glavaš · 2021
Earlier work this paper cites.
Towards understanding and mitigating social biases in language models
Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov · 2021
Earlier work this paper cites.

Similar

Are nlp models really able to solve simple math word problems?
Original
Arkil Patel, Satwik Bhattamishra, and Navin Goyal · 2021
Cited alongside, same era.
Challenges in detoxifying language models
Original
Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang · 2021
Cited alongside, same era.
Chatgpt usage and limitations
Amos Azaria · 2022
Cited alongside, same era.
Pros and cons of gan evaluation measures: New developments
Ali Borji · 2022
Cited alongside, same era.
Palm: Scaling language modeling with pathways
Original
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al · 2022
Cited alongside, same era.
Changing the nature of ai research
Subbarao Kambhampati · 2022
Cited alongside, same era.
Large language models are zero-shot reasoners
Original
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa · 2022
Cited alongside, same era.
Is gpt-3 a psychopath? evaluating large language models from a psychological perspective
Original
Xingxuan Li, Yutong Li, Linlin Liu, Lidong Bing, and Shafiq Joty · 2022
Cited alongside, same era.
Competition-level code generation with alphacode
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al · 2022
Cited alongside, same era.
Can large language models reason about medical questions?
Original
Valentin Liévin, Christoffer Egeberg Hother, and Ole Winther · 2022
Cited alongside, same era.
Bayesformer: Transformer with uncertainty estimation
Original
Karthik Abinav Sankararaman, Sinong Wang, and Han Fang · 2022
Cited alongside, same era.
Language models are greedy reasoners: A systematic formal analysis of chain-of-thought
Original
Abulhair Saparov and He He · 2022
Cited alongside, same era.

Then

Mathematical capabilities of chatgpt, 2023
Simon Frieder, Luca Pinchetti, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewicz, Philipp Christian Petersen, Alexis Chevalier, and Julius Berner · 2023
Closest in time.
How close is chatgpt to human experts? comparison corpus, evaluation, and detection
Original
Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu · 2023
Closest in time.
The political ideology of conversational ai: Converging evidence on chatgpt’s pro-environmental, left-libertarian orientation
Original
Jochen Hartmann, Jasper Schwenzow, and Maximilian Witte · 2023
Closest in time.
How good are gpt models at machine translation? a comprehensive evaluation
Original
Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla · 2023
Closest in time.
Is chatgpt a good translator? a preliminary study
Original
Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, and Zhaopeng Tu · 2023
Closest in time.
A watermark for large language models
Original
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein · 2023
Closest in time.
Could an artificial-intelligence agent pass an introductory physics course?
Original
Gerd Kortemeyer · 2023
Closest in time.
Theory of mind may have spontaneously emerged in large language models
Original
Michal Kosinski · 2023
Closest in time.
The moral authority of chatgpt
Original
Sebastian Krügel, Andreas Ostermaier, and Matthias Uhl · 2023
Closest in time.
Machine love
Original
Joel Lehman · 2023
Closest in time.
Dissociating language and thought in large language models: a cognitive perspective
Original
Kyle Mahowald, Anna A Ivanova, Idan A Blank, Nancy Kanwisher, Joshua B Tenenbaum, and Evelina Fedorenko · 2023
Closest in time.
How generative ai models such as chatgpt can be (mis)used in spc practice, education, and research? an exploratory study, 2023
Fadel M. Megahed, Ying-Ju Chen, Joshua A. Ferris, Sven Knoth, and L. Allison Jones-Farmer · 2023
Closest in time.
Putting chatgpt’s medical advice to the (turing) test
Oded Nov, Nina Singh, and Devin M Mann · 2023
Closest in time.
Linguistic ambiguity analysis in chatgpt
Original
Miguel Ortega-Martín, Óscar García-Sierra, Alfonso Ardoiz, Jorge Álvarez, Juan Carlos Armenteros, and Adrián Alonso · 2023
Closest in time.
Is chatgpt a general-purpose natural language processing task solver?
Original
Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang · 2023
Closest in time.
Can ai-generated text be reliably detected?
Original
Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi · 2023
Closest in time.
An analysis of the automatic bug fixing performance of chatgpt
Original
Dominik Sobania, Martin Briesch, Carol Hanna, and Justyna Petke · 2023
Closest in time.
What chatgpt and generative ai mean for science
Chris Stokel-Walker and Richard Van Noorden · 2023
Closest in time.
Chatgpt: five priorities for research
Eva AM van Dis, Johan Bollen, Willem Zuidema, Robert van Rooij, and Claudi L Bockting · 2023
Closest in time.
Exploring the limits of chatgpt for query or aspect-based text summarization
Original
Xianjun Yang, Yan Li, Xinlu Zhang, Haifeng Chen, and Wei Cheng · 2023
Closest in time.
A comprehensive survey on pretrained foundation models: A history from bert to chatgpt
Original
Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, et al · 2023
Closest in time.
Exploring ai ethics of chatgpt: A diagnostic analysis
Original
Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…