Can ChatGPT Play the Role of a Teaching Assistant in an Introductory Programming Course?

Elements of Software Science (Operating and Programming Systems Series)

Maurice H. Halstead. 1977 · 1977

Earlier work this paper cites.

Automatic Grading of Programming Assignments: An Approach Based on Formal Semantics. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) . 126–137

Xiao Liu, Shuai Wang, Pei Wang, and Dinghao Wu. 2019 · 2019

Earlier work this paper cites.

Introducing ChatGPT

OpenAI. 2022 · 2022

Earlier work this paper cites.

Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models. In Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 1 (Lugano and Virtual Event, Switzerland) (ICER ’22) . Association for Computing Machinery, New York, NY, USA, 27–43

Sami Sarsa, Paul Denny, Arto Hellas, and Juho Leinonen. 2022 · 2022

Earlier work this paper cites.

Investigating the Potential of GPT-3 in Providing Feedback for Programming Assessments. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (Turku, Finland) (ITiCSE 2023) . Association for Computing Machinery, New York, NY, USA, 292–298

Rishabh Balse, Bharath Valaboju, Shreya Singhal, Jayakrishnan Madathil Warriem, and Prajish Prasad. 2023 · 2023

Earlier work this paper cites.

Programming Is Hard - Or at Least It Used to Be: Educational Opportunities and Challenges of AI Code Generation. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (Toronto ON, Canada) (SIGCSE 2023) . Association for Computing Machinery, New York, NY, USA, 500–506

Brett A. Becker, Paul Denny, James Finnie-Ansley, Andrew Luxton-Reilly, James Prather, and Eddie Antonio Santos. 2023 · 2023

Earlier work this paper cites.

GPT-3 vs Object Oriented Programming Assignments: An Experience Report. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (Turku, Finland) (ITiCSE 2023) . Association for Computing Machinery, New York, NY, USA, 61–67

Bruno Pereira Cipriano and Pedro Alves. 2023 · 2023

Earlier work this paper cites.

How ChatGPT Will Change Software Engineering Education. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (Turku, Finland) (ITiCSE 2023) . Association for Computing Machinery, New York, NY, USA, 110–116

Marian Daun and Jennifer Brings. 2023 · 2023

Earlier work this paper cites.

Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (Toronto ON, Canada) (SIGCSE 2023) . Association for Computing Machinery, New York, NY, USA, 1136–1142

Paul Denny, Viraj Kumar, and Nasser Giacaman. 2023 · 2023

Earlier work this paper cites.

Concept-Based Automated Grading of CS-1 Programming Assignments. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (, Seattle, WA, USA,) (ISSTA 2023) . Association for Computing Machinery, New York, NY, USA, 199–210

Zhiyu Fan, Shin Hwei Tan, and Abhik Roychoudhury. 2023 · 2023

Earlier work this paper cites.

My AI Wants to Know If This Will Be on the Exam: Testing OpenAI’s Codex on CS2 Programming Exercises. In Proceedings of the 25th Australasian Computing Education Conference (Melbourne, VIC, Australia) (ACE ’23) . Association for Computing Machinery, New York, NY, USA, 97–104

James Finnie-Ansley, Paul Denny, Andrew Luxton-Reilly, Eddie Antonio Santos, James Prather, and Brett A. Becker. 2023 · 2023

Cited alongside, same era.

Experiences with TA-Bot in CS1. In Proceedings of the ACM Conference on Global Computing Education Vol 1 (, Hyderabad, India,) (CompEd 2023) . Association for Computing Machinery, New York, NY, USA, 57–63

Jack Forden, Alexander Gebhard, and Dennis Brylow. 2023 · 2023

Cited alongside, same era.

Exploring the Responses of Large Language Models to Beginner Programmers’ Help Requests. In Proceedings of the 2023 ACM Conference on International Computing Education Research V.1 (ICER 2023) . ACM

Arto Hellas, Juho Leinonen, Sami Sarsa, Charles Koutcheme, Lilja Kujanpää, and Juha Sorva. 2023 · 2023

Cited alongside, same era.

AI-TA: Towards an Intelligent Question-Answer Teaching Assistant using Open-Source LLMs

GPT-4 as an Automatic Grader : The accuracy of grades set by GPT-4 on introductory programming assignments

Filippa Nilsson and Jonatan Tuvstedt. 2023 · 2023

Closest in time.

OpenAI Codex

OpenAI. 2023 · 2023

Closest in time.

ChatGPT, Can You Generate Solutions for My Coding Exercises? An Evaluation on Its Effectiveness in an Undergraduate Java Programming Course.. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (Turku, Finland) (ITiCSE 2023) . Association for Computing Machinery, New York, NY, USA, 54–60

Eng Lieh Ouh, Benjamin Kok Siew Gan, Kyong Jin Shim, and Swavek Wlodkowski. 2023 · 2023

Closest in time.

Evaluating the Performance of Code Generation Models for Solving Parsons Problems With Small Prompt Variations. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (Turku, Finland) (ITiCSE 2023) . Association for Computing Machinery, New York, NY, USA, 299–305

Brent Reeves, Sami Sarsa, James Prather, Paul Denny, Brett A. Becker, Arto Hellas, Bailey Kimmel, Garrett Powell, and Juho Leinonen. 2023 · 2023