Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machina

What computers can’t do: The limits of artificial intelligence

Dreyfus, Hubert L. 1972 · 1972

Earlier work this paper cites.

Back to the future: Towards explainable temporal reasoning with large language models

Yuan, Chenhan, Qianqian Xie, Jimin Huang, Sophia Ananiadou. 2024 · 1974

Earlier work this paper cites.

Minds, brains, and programs

Searle, John R. 1980 · 1980

Earlier work this paper cites.

The mind’s i: Fantasies and reflections on self and soul

Hofstadter, Douglas R, Daniel Clement Dennett. 1981 · 1981

Earlier work this paper cites.

The philosophy of artificial intelligence,«oxford readings in philosophy»

Boden, Margaret. 1992 · 1992

Earlier work this paper cites.

Scaling laws for neural language models

Original

Kaplan, Jared, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, Dario Amodei. 2020 · 2001

Earlier work this paper cites.

Language models are few-shot learners

Original

Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Ma teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei. 2020 · 2005

Earlier work this paper cites.

Who thinks about the competition? managerial ability and strategic entry in us local telephone markets

Goldfarb, Avi, Mo Xiao. 2011 · 2011

Earlier work this paper cites.

The 11–20 money request game: A level-k reasoning study

Arad, Ayala, Ariel Rubinstein. 2012 · 2012

Earlier work this paper cites.

Structural models of nonequilibrium strategic thinking: Theory, evidence, and applications

Crawford, Vincent P, Miguel A Costa-Gomes, Nagore Iriberri. 2013 · 2013

Earlier work this paper cites.

Universal language model fine-tuning for text classification

Original

Howard, Jeremy, Sebastian Ruder. 2018 · 2018

Earlier work this paper cites.

The secret sharer: Evaluating and testing unintended memorization in neural networks

Carlini, Nicholas, Chang Liu, Úlfar Erlingsson, Jernej Kos, Dawn Song. 2019 · 2019

Earlier work this paper cites.

Retrieval-augmented generation for knowledge-intensive nlp tasks

Lewis, Patrick, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020 · 2020

Earlier work this paper cites.

Extracting training data from large language models

Carlini, Nicholas, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021 · 2021

Earlier work this paper cites.

Understanding unintended memorization in language models under federated learning

Thakkar, Om Dipakbhai, Swaroop Ramaswamy, Rajiv Mathews, Francoise Beaufays. 2021 · 2021

Earlier work this paper cites.

Rethinking the role of demonstrations: What makes in-context learning work?

Original

Min, Sewon, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, Luke Zettlemoyer. 2022 · 2022

Earlier work this paper cites.

Chain-of-thought prompting elicits reasoning in large language models

Wei, Jason, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022 · 2022

Earlier work this paper cites.

Gpt-4 technical report

Original

Achiam, Josh, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023 · 2023

Earlier work this paper cites.

Using large language models to simulate multiple humans and replicate human subject studies

Aher, Gati V, Rosa I Arriaga, Adam Tauman Kalai. 2023 · 2023

Earlier work this paper cites.

Out of one, many: Using language models to simulate human samples

Argyle, Lisa P, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, David Wingate. 2023 · 2023

Earlier work this paper cites.

A systematic evaluation of large language models on out-of-distribution logical reasoning tasks

Original

Bao, Qiming, Gaël Gendron, Alex Yuxuan Peng, Wanjun Zhong, Neset Tan, Yang Chen, Michael Witbrock, Jiamou Liu. 2023 · 2023

Earlier work this paper cites.

The reversal curse: Llms trained on" a is b" fail to learn" b is a"

Original

Berglund, Lukas, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, Owain Evans. 2023 · 2023

Earlier work this paper cites.

Using gpt for market research

Brand, James, Ayelet Israeli, Donald Ngwe. 2023 · 2023

Earlier work this paper cites.

Playing games with gpt: What can we learn about a large language model from canonical strategic games?

Brookins, Philip, Jason Matthew DeBacker. 2023 · 2023

Earlier work this paper cites.

Rethink reporting of evaluation results in ai

Burnell, Ryan, Wout Schellaert, John Burden, Tomer D Ullman, Fernando Martinez-Plumed, Joshua B Tenenbaum, Danaja Rutar, Lucy G Cheke, Jascha Sohl-Dickstein, Melanie Mitchell, et al. 2023 · 2023

Earlier work this paper cites.

Consciousness in artificial intelligence: insights from the science of consciousness

Original

Butlin, Patrick, Robert Long, Eric Elmoznino, Yoshua Bengio, Jonathan Birch, Axel Constant, George Deane, Stephen M Fleming, Chris Frith, Xu Ji, et al. 2023 · 2023

Earlier work this paper cites.

Open problems and fundamental limitations of reinforcement learning from human feedback

Original

Casper, Stephen, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, et al. 2023 · 2023

Earlier work this paper cites.

Llms differ from human cognition because they are not embodied

Chemero, Anthony. 2023 · 2023

Cited alongside, same era.

Can ai language models replace human participants?

Dillion, Danica, Niket Tandon, Yuling Gu, Kurt Gray. 2023 · 2023

Cited alongside, same era.

Large language models are not strong abstract reasoners

Original

Gendron, Gaël, Qiming Bao, Michael Witbrock, Gillian Dobbie. 2023 · 2023

Cited alongside, same era.

Ai and the transformation of social science research

Grossmann, Igor, Matthew Feinberg, Dawn C Parker, Nicholas A Christakis, Philip E Tetlock, William A Cunningham. 2023 · 2023

Cited alongside, same era.

Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods

Original

Hagendorff, Thilo. 2023 · 2023

Cited alongside, same era.

Generalization or memorization: Data contamination and trustworthy evaluation for large language models

Original

Dong, Yihong, Xue Jiang, Huanyu Liu, Zhi Jin, Ge Li. 2024 · 2024

Closest in time.

Frontiers: Can large language models capture human preferences?

Goli, Ali, Amandeep Singh. 2024 · 2024

Closest in time.

Economics arena for large language models

Original

Guo, Shangmin, Haoran Bu, Haochuan Wang, Yi Ren, Dianbo Sui, Yuming Shang, Siting Lu. 2024 · 2024

Closest in time.

Changing answer order can decrease mmlu accuracy

Original

Gupta, Vipul, David Pantoja, Candace Ross, Adina Williams, Megan Ung. 2024 · 2024

Closest in time.

How does chatgpt’think’? psychology and neuroscience crack open ai large language models

Hutson, Matthew. 2024 · 2024

Closest in time.