Can we trust the evaluation on ChatGPT?

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901

Earlier work this paper cites.

Catastrophic interference in connectionist networks: The sequential learning problem

Michael McCloskey and Neal J Cohen. 1989 · 1989

Earlier work this paper cites.

Semeval-2016 task 6: Detecting stance in tweets

Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016 · 2016

Earlier work this paper cites.

Stance and sentiment in tweets

Saif M Mohammad, Parinaz Sobhani, and Svetlana Kiritchenko. 2017 · 2017

Earlier work this paper cites.

Proximal policy optimization algorithms

Original

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017 · 2017

Earlier work this paper cites.

Language models are unsupervised multitask learners

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019 · 2019

Earlier work this paper cites.

Stance detection: A survey

Dilek Küçük and Fazli Can. 2020 · 2020

Earlier work this paper cites.

On the dangers of stochastic parrots: Can language models be too big?

Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021 · 2021

Earlier work this paper cites.

Extracting training data from large language models

Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B Brown, Dawn Song, Ulfar Erlingsson, et al. 2021 · 2021

Earlier work this paper cites.

Documenting large webtext corpora: A case study on the colossal clean crawled corpus

Original

Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. 2021 · 2021

Earlier work this paper cites.

P-stance: A large dataset for stance detection in political domain

Yingjie Li, Tiberiu Sosea, Aditya Sawant, Ajith Jayaraman Nair, Diana Inkpen, and Cornelia Caragea. 2021 · 2021

Cited alongside, same era.

What can transformers learn in-context? a case study of simple function classes

Original

Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. 2022 · 2022

Cited alongside, same era.

Rethinking the role of demonstrations: What makes in-context learning work?

Original

Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022 · 2022

Cited alongside, same era.

Chatgpt: Optimizing language models for dialogue

OpenAI. 2022 · 2022

Cited alongside, same era.

Beyond the imitation game: Quantifying and extrapolating the capabilities of language models

Original

Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022 · 2022

Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models

Tiffany H Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, et al. 2023 · 2023

Closest in time.

Gpt-4 technical report

Original

OpenAI. 2023 · 2023

Closest in time.

Chatgpt: the future of discharge summaries?

Sajan B Patel and Kyle Lam. 2023 · 2023

Closest in time.

An analysis of the automatic bug fixing performance of chatgpt

Original

Dominik Sobania, Martin Briesch, Carol Hanna, and Justyna Petke. 2023 · 2023

Closest in time.

What chatgpt and generative ai mean for science

Chris Stokel-Walker and Richard Van Noorden. 2023 · 2023

Closest in time.