ChatGPT Participates in a Computer Science Exam

Language models are few-shot learners

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al · 2020

Earlier work this paper cites.

Gpt-3, bloviator: Openai’s language generator has no idea what it’s talking about

G. Marcus and E. Davis · 2020

Earlier work this paper cites.

The chess transformer: Mastering play using generative language models

Original

D. Noever, M. Ciolino, and J. Kalin · 2020

Earlier work this paper cites.

On the dangers of stochastic parrots: Can language models be too big?

E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell · 2021

Earlier work this paper cites.

On the opportunities and risks of foundation models

Original

R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al · 2021

Earlier work this paper cites.

Evaluating large language models trained on code

Original

M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al · 2021

Earlier work this paper cites.

Ammus: A survey of transformer-based pretrained models in natural language processing

Original

K. S. Kalyan, A. Rajasekharan, and S. Sangeetha · 2021

Earlier work this paper cites.

Do prompt-based models really understand the meaning of their prompts?

Original

A. Webson and E. Pavlick · 2021

Earlier work this paper cites.

GPT takes the bar exam

Original

M. Bommarito and D. M. Katz · 2022

Earlier work this paper cites.

A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level

I. Drori, S. Zhang, R. Shuttleworth, L. Tang, A. Lu, E. Ke, K. Liu, L. Chen, S. Tran, N. Cheng, et al · 2022

Cited alongside, same era.

Training language models to follow instructions with human feedback

Original

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al · 2022

Cited alongside, same era.

Beyond the imitation game: Quantifying and extrapolating the capabilities of language models

Original

A. Srivastava, A. Rastogi, A. Rao, A. A. M. Shoeb, A. Abid, A. Fisch, A. R. Brown, A. Santoro, A. Gupta, A. Garriga-Alonso, et al · 2022

Cited alongside, same era.

Chain of thought prompting elicits reasoning in large language models

Original

J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou · 2022

Cited alongside, same era.

Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models

T. H. Kung, M. Cheatham, A. Medenilla, C. Sillos, L. De Leon, C. Elepaño, M. Madriaga, R. Aggabao, G. Diaz-Candido, J. Maningo, et al · 2023

Closest in time.

Chatgpt: A meta-analysis after 2.5 months

Original

C. Leiter, R. Zhang, Y. Chen, , J. Belouadi, D. Larionov, V. Fresen, and S. Eger · 2023

Closest in time.

Learning performance-improving code edits

Original

A. Madaan, A. Shypula, U. Alon, M. Hashemi, P. Ranganathan, Y. Yang, G. Neubig, and A. Yazdanbakhsh · 2023

Closest in time.

Did chatgpt really pass graduate-level exams?

M. Mitchell · 2023

Closest in time.

Gpt-4 and professional benchmarks: the wrong answer to the wrong question

A. Narayanan and S. Kapoor · 2023

Closest in time.