How is ChatGPT's behavior changing over time?

Unsolved problems in number theory

Richard Guy · 2004

Earlier work this paper cites.

Hotpotqa: A dataset for diverse, explainable multi-hop question answering

Original

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning · 2018

Earlier work this paper cites.

Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task

Original

Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al · 2018

Earlier work this paper cites.

On the measure of intelligence

Original

François Chollet · 2019

Earlier work this paper cites.

Realtoxicityprompts: Evaluating neural toxic degeneration in language models

Original

Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith · 2020

Earlier work this paper cites.

Program synthesis with large language models

Original

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al · 2021

Earlier work this paper cites.

Did the model change? efficiently assessing machine learning api shifts

Original

Lingjiao Chen, Tracy Cai, Matei Zaharia, and James Zou · 2021

Earlier work this paper cites.

Evaluating large language models trained on code

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, et al · 2021

Earlier work this paper cites.

Extracting training data from large language models

Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al · 2021

Earlier work this paper cites.

Hapi: A large-scale longitudinal dataset of commercial ml api predictions

Lingjiao Chen, Zhihua Jin, Evan Sabri Eyuboglu, Christopher Ré, Matei Zaharia, and James Y Zou · 2022

Earlier work this paper cites.

News summarization and evaluation in the era of gpt-3

Original

Tanya Goyal, Junyi Jessy Li, and Greg Durrett · 2022

Cited alongside, same era.

Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned

Original

Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al · 2022

Cited alongside, same era.

Holistic evaluation of language models

Original

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al · 2022

Cited alongside, same era.

Chain of thought prompting elicits reasoning in large language models

Original

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou · 2022

Cited alongside, same era.

Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models

Tiffany H Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, et al · 2023

Closest in time.

Evaluating the logical reasoning ability of chatgpt and gpt-4

Original

Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, Jian Liu, Qiji Zhou, and Yue Zhang · 2023

Closest in time.

Is GPT-4 getting worse over time?

Arvind Narayanan and Sayash Kapoor · 2023

Closest in time.

Capabilities of gpt-4 on medical challenge problems

Original

Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, and Eric Horvitz · 2023

Closest in time.

Whose opinions do language models reflect?

Original

Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto · 2023