How do physics students evaluate artificial intelligence responses on comprehension questions? A study on the perceived scientific accuracy and linguistic quality

J. Weizenbaum, ELIZA — a computer program for the study of natural language communication between man and machine, Communications of the ACM 9

1966

Earlier work this paper cites.

J. A. Colliver, S. J. Verhulst, and H. S. Barrows, Self-assessment in medical practice: A further concern about the conventional research paradigm, Teaching and Learning in Medicine 17

2005

Earlier work this paper cites.

C. S. Große and A. Renkl, Finding and fixing errors in worked examples: Can this foster learning outcomes?, Learning and Instruction 17

2007

Earlier work this paper cites.

A. Mason and C. Singh, Do advanced physics students learn from their mistakes without explicit intervention?, American Journal of Physics 78

2010

Earlier work this paper cites.

A. Prinz, S. Golke, and J. Wittwer, The double curse of misconceptions: misconceptions impair not only text comprehension but also metacomprehension in the domain of statistics, Instructional Science 46

2018

Earlier work this paper cites.

S. Chatterjee and K. K. Bhattacharjee, Adoption of artificial intelligence in higher education: a quantitative analysis using structural equation modelling, Education and Information Technologies 25

2020

Earlier work this paper cites.

C. W. Okonkwo and A. Ade-Ibijola, Chatbots applications in education: A systematic review, Computers and Education: Artificial Intelligence 2

2021

Earlier work this paper cites.

A. Gilson, C. Safranek, T. Huang, V. Socrates, L. Chi, R. A. Taylor, and D. Chartash, How Does ChatGPT Perform on the Medical Licensing Exams? The Implications of Large Language Models for Medical Education and Knowledge Assessment (2022), medRxiv 2022.12.23.22283901

2022

Earlier work this paper cites.

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, Training language models to follow instructions with human feedback (2022), arXiv:2203.02155 [cs]

Original

2022

Earlier work this paper cites.

S. Lock, What is AI chatbot phenomenon ChatGPT and could it replace humans?, The Guardian (2022)

2022

Cited alongside, same era.

K. Roose, The Brilliance and Weirdness of ChatGPT, The New York Times (2022)

2022

Cited alongside, same era.

P. Verma and R. Lerman, What is ChatGPT, the viral social media AI?, The Washington Post (2022) , section: Innovations

2022

Cited alongside, same era.

https://chat.openai.com/ , accessed on 21/02/2023

2023

Cited alongside, same era.

OpenAI, ChatGPT: Optimizing Language Models for Dialogue (2022a), accessed on 12/04/2023

2023

Cited alongside, same era.

J. Holmes, Z. Liu, L. Zhang, Y. Ding, T. T. Sio, L. A. McGee, J. B. Ashman, X. Li, T. Liu, J. Shen, and W. Liu, Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physics (2023), arXiv:2304.01938 [physics]

Original

2023

Closest in time.

R. Chocarro, M. Cortiñas, and G. Marcos-Matás, Teachers’ attitudes towards chatbots in education: a technology acceptance model approach considering the effect of social language, bot proactiveness, and users’ characteristics, Educational Studies 49

2023

Closest in time.

OpenAI, ChatGPT — Release Notes (2022b), accessed on 12/04/2023

2023

Closest in time.

OpenAI, GPT-4 Technical Report (2023), arXiv:2303.08774 [cs]

Original

2023

Closest in time.

S. Fergus, M. Botha, and M. Ostovar, Evaluating Academic Answers Generated Using ChatGPT, Journal of Chemical Education 100