LLMs May Not Be Human-Level Players, But They Can Be Testers: Measuring Game Difficulty with LLM Agents

Flow: The psychology of optimal experience

Mihaly Czikszentmihalyi. 1990 · 1990

Earlier work this paper cites.

The game, the player, the world: Looking for a heart of gameness. In Proceedings of DiGRA 2003 Conference: Level Up

Jesper Juul. 2003 · 2003

Earlier work this paper cites.

Andrew Rollings and Ernest Adams on game design

Andrew Rollings and Ernest Adams. 2003 · 2003

Earlier work this paper cites.

Game design through self-play experiments. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology . 421–428

Alasdair Macleod. 2005 · 2005

Earlier work this paper cites.

The game master. In ACM International Conference Proceeding Series , Vol. 123. 215–222

Anders Tychsen, Michael Hitchens, Thea Brolund, and Manolya Kavakli. 2005 · 2005

Earlier work this paper cites.

Scaling the level of difficulty in single player video games. In Entertainment Computing–ICEC 2009: 8th International Conference, Paris, France, September 3-5, 2009. Proceedings 8 . Springer, 24–35

Maria-Virginia Aponte, Guillaume Levieux, and Stéphane Natkin. 2009 · 2009

Earlier work this paper cites.

Measuring the level of difficulty in single player video games

Maria-Virginia Aponte, Guillaume Levieux, and Stephane Natkin. 2011 · 2011

Earlier work this paper cites.

Introduction to game AI

Neil Kirby and Heather Hurley. 2011 · 2011

Earlier work this paper cites.

The art of failure: An essay on the pain of playing video games

Jesper Juul. 2013 · 2013

Earlier work this paper cites.

Playing atari with deep reinforcement learning

Original

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013 · 2013

Earlier work this paper cites.

A survey of real-time strategy game AI research and competition in StarCraft

Santiago Ontanón, Gabriel Synnaeve, Alberto Uriarte, Florian Richoux, David Churchill, and Mike Preuss. 2013 · 2013

Earlier work this paper cites.

AI-based game design patterns

Mike Treanor, Alexander Zook, Mirjam P Eladhari, Julian Togelius, Gillian Smith, Michael Cook, Tommy Thompson, Brian Magerko, John Levine, and Adam Smith. 2015 · 2015

Earlier work this paper cites.

Deep reinforcement learning from self-play in imperfect-information games

Original

Johannes Heinrich and David Silver. 2016 · 2016

Earlier work this paper cites.

General video game ai: Competition, challenges and opportunities. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 30

Diego Perez-Liebana, Spyridon Samothrakis, Julian Togelius, Tom Schaul, and Simon Lucas. 2016 · 2016

Earlier work this paper cites.

Playing FPS games with deep reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence , Vol. 31

Guillaume Lample and Devendra Singh Chaplot. 2017 · 2017

Earlier work this paper cites.

Using a team of general ai algorithms to assist game design and testing. In 2018 IEEE Conference on Computational Intelligence and Games (CIG) . IEEE, 1–8

Cristina Guerrero-Romero, Simon M Lucas, and Diego Perez-Liebana. 2018 · 2018

Earlier work this paper cites.

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al · 2018

Earlier work this paper cites.

Improving hearthstone ai by combining mcts and supervised learning algorithms. In 2018 IEEE conference on computational intelligence and games (CIG) . IEEE, 1–8

Maciej Świechowski, Tomasz Tajmajer, and Andrzej Janusz. 2018 · 2018

Earlier work this paper cites.

Improving RTS game AI by supervised policy learning, tactical search, and deep reinforcement learning

Nicolas A Barriga, Marius Stanescu, Felipe Besoain, and Michael Buro. 2019 · 2019

Earlier work this paper cites.

Friend, collaborator, student, manager: How design of an ai-driven game level editor affects creators. In Proceedings of the 2019 CHI conference on human factors in computing systems . 1–13

Matthew Guzdial, Nicholas Liao, Jonathan Chen, Shao-Yu Chen, Shukan Shah, Vishwa Shah, Joshua Reno, Gillian Smith, and Mark O Riedl. 2019 · 2019

Earlier work this paper cites.

Implicit communication of actionable information in human-ai teams. In Proceedings of the 2019 CHI conference on human factors in computing systems . 1–13

Claire Liang, Julia Proft, Erik Andersen, and Ross A Knepper. 2019 · 2019

Earlier work this paper cites.

Slay the Spire

MegaCrit. 2019 · 2019

Cited alongside, same era.

A survey of deep reinforcement learning in video games

Original

Kun Shao, Zhentao Tang, Yuanheng Zhu, Nannan Li, and Dongbin Zhao. 2019 · 2019

Cited alongside, same era.

Grandmaster level in StarCraft II using multi-agent reinforcement learning

Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al · 2019

Cited alongside, same era.

From chess and atari to starcraft and beyond: How game ai is driving the world of ai

Sebastian Risi and Mike Preuss. 2020 · 2020

Cited alongside, same era.

Supervised learning achieves human-level performance in moba games: A case study of honor of kings

Deheng Ye, Guibin Chen, Peilin Zhao, Fuhao Qiu, Bo Yuan, Wen Zhang, Sheng Chen, Mingfei Sun, Xiaoqian Li, Siqin Li, et al · 2020

Cited alongside, same era.

Language agents with reinforcement learning for strategic play in the werewolf game

Original

Zelai Xu, Chao Yu, Fei Fang, Yu Wang, and Yi Wu. 2023 · 2023

Later among the works it cites.

CALYPSO: LLMs as Dungeon Master’s Assistants. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment , Vol. 19. 380–390

Andrew Zhu, Lara Martin, Andrew Head, and Chris Callison-Burch. 2023 · 2023

Later among the works it cites.

Language-Driven Play: Large Language Models as Game-Playing Agents in Slay the Spire. In Proceedings of the 19th International Conference on the Foundations of Digital Games . 1–10

Bahar Bateni and Jim Whitehead. 2024 · 2024

Closest in time.

Wordle Stats – How Hard is Today’s Wordle?

Chris. 2024 · 2024

Closest in time.

Chessgpt: Bridging policy learning and language modeling

Xidong Feng, Yicheng Luo, Ziyan Wang, Hongrui Tang, Mengyue Yang, Kun Shao, David Mguni, Yali Du, and Jun Wang. 2024 · 2024