EconEvals: Benchmarks and Litmus Tests for Economic Decision-Making by LLM Agents

College Admissions and the Stability of Marriage

D. Gale and L. S. Shapley · 1962

Earlier work this paper cites.

Marriages stables

Donald E. Knuth · 1976

Earlier work this paper cites.

Estimating Discrete-Choice Models of Product Differentiation

Steven T. Berry · 1994

Earlier work this paper cites.

Are emily and greg more employable than lakisha and jamal? a field experiment on labor market discrimination

Marianne Bertrand and Sendhil Mullainathan · 2004

Earlier work this paper cites.

On the complexity of trial and error

Xiaohui Bei, Ning Chen, and Shengyu Zhang · 2013

Earlier work this paper cites.

Global evidence on economic preferences*

Armin Falk, Anke Becker, Thomas Dohmen, Benjamin Enke, David Huffman, and Uwe Sunde · 2018

Earlier work this paper cites.

Developing competition law for collusion by autonomous artificial agents

Joseph E Harrington, Jr · 2018

Earlier work this paper cites.

The Complexity of Interactively Learning a Stable Matching by Trial and Error

Ehsan Emamjomeh-Zadeh, Yannai A. Gonczarowski, and David Kempe · 2020

Earlier work this paper cites.

On the fairness of machine-assisted human decisions, 2021

Original

Talia Gillis, Bryce McLaughlin, and Jann Spiess · 2021

Earlier work this paper cites.

Testing the waters: Behavior across participant pools

Erik Snowberg and Leeat Yariv · 2021

Earlier work this paper cites.

Artificial intelligence and auction design

Original

Martino Banchio and Andrzej Skrzypacz · 2022

Earlier work this paper cites.

Algorithmic design: Fairness versus accuracy

Annie Liang, Jay Lu, and Xiaosheng Mu · 2022

Earlier work this paper cites.

Using large language models to simulate multiple humans and replicate human subject studies

Original

Gati Aher, Rosa I. Arriaga, and Adam Tauman Kalai · 2023

Earlier work this paper cites.

Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?, April 2023

John J. Horton · 2023

Earlier work this paper cites.

Frontiers: Can Large Language Models Capture Human Preferences?

Ali Goli and Amandeep Singh · 2023

Earlier work this paper cites.

Adaptive algorithms and collusion via coupling

Martino Banchio and Giacomo Mantegazza · 2023

Earlier work this paper cites.

Inverse selection, 2023

Markus K Brunnermeier, Rohit Lamba, and Carlos Segura-Rodriguez · 2023

Earlier work this paper cites.

Generative AI at Work, April 2023

Erik Brynjolfsson, Danielle Li, and Lindsey R. Raymond · 2023

Earlier work this paper cites.

The market effects of algorithms

Lindsey Raymond · 2023

Earlier work this paper cites.

Adversarial competition and collusion in algorithmic markets

Luc Rocher, Arnaud J Tournier, and Yves-Alexandre de Montjoye · 2023

Earlier work this paper cites.

FinanceBench: A New Benchmark for Financial Question Answering, November 2023

Original

Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen · 2023

Earlier work this paper cites.

Voyager: An Open-Ended Embodied Agent with Large Language Models, October 2023

Original

Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar · 2023

Earlier work this paper cites.

GAIA: a benchmark for General AI Assistants

Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom · 2023

Earlier work this paper cites.

WebArena: A Realistic Web Environment for Building Autonomous Agents

Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig · 2023

Earlier work this paper cites.

AgentBench: Evaluating LLMs as Agents

Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang · 2023

Earlier work this paper cites.

PlanBench: an extensible benchmark for evaluating large language models on planning and reasoning about change

Original

Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati · 2023

Cited alongside, same era.

Do the rewards justify the means? measuring trade-offs between rewards and ethical behavior in the machiavelli benchmark

Original

Alexander Pan, Jun Shern Chan, Andy Zou, Nathaniel Li, Steven Basart, Thomas Woodside, Hanlin Zhang, Scott Emmons, and Dan Hendrycks · 2023

Cited alongside, same era.

Toolformer: Language Models Can Teach Themselves to Use Tools

Original

Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom · 2023

Cited alongside, same era.

Welfare Distribution in Two-sided Random Matching Markets

Itai Ashlagi, Mark Braverman, and Geng Zhao · 2023

Cited alongside, same era.

Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting

Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr · 2024

Later among the works it cites.

ProSA: Assessing and understanding the prompt sensitivity of LLMs

Jingming Zhuo, Songyang Zhang, Xinyu Fang, Haodong Duan, Dahua Lin, and Kai Chen · 2024

Later among the works it cites.

Regulation of algorithmic collusion

Jason D. Hartline, Sheng Long, and Chenhao Zhang · 2024

Later among the works it cites.

Algorithmic collusion: where are we and where should we be going?, August 2024

Ibrahim Abada, Joseph E Harrington, Jr, Xavier Lambin, and Janusz M Meylahn · 2024

Later among the works it cites.

AI capabilities have steadily improved over the past year

Luke Emberson · 2025

Closest in time.

Visa Launches AI Agents for Shopping, May 2025