EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning

Theory of Games and Economic Behavior (60th Anniversary Commemorative Edition)

John von Neumann, Oskar Morgenstern, and Ariel Rubinstein. 1944 · 1944

Earlier work this paper cites.

Policy gradient methods for reinforcement learning with function approximation

Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999 · 1999

Earlier work this paper cites.

Simple statistical gradient-following algorithms for connectionist reinforcement learning

Ronald J. Williams. 2004 · 2004

Earlier work this paper cites.

Deep reinforcement learning from human preferences

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017 · 2017

Earlier work this paper cites.

Proximal policy optimization algorithms

Original

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017 · 2017

Earlier work this paper cites.

Alfred: A benchmark for interpreting grounded instructions for everyday tasks

Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020 · 2020

Earlier work this paper cites.

A minimalist approach to offline reinforcement learning

Scott Fujimoto and Shixiang Shane Gu. 2021 · 2021

Earlier work this paper cites.

Logiqa: a challenge dataset for machine reading comprehension with logical reasoning

Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2021 · 2021

Earlier work this paper cites.

{ALFW}orld: Aligning text and embodied environments for interactive learning

Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Cote, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2021 · 2021

Earlier work this paper cites.

Human-level play in the game of diplomacy by combining language models with strategic reasoning

Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, Athul Paul Jacob, Mojtaba Komeili, Karthik Konath, Minae Kwon, Adam Lerer, Mike Lewis, Alexander H. Miller, Sandra Mitts, Adithya Renduchintala, Stephen Roller, Dirk Rowe, Weiyan Shi, Joe Spisak, Alexander Wei, David J. Wu, Hugh Zhang, and Markus Zijlstra. 2022 · 2022

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022

Earlier work this paper cites.

Chain of thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022 · 2022

Earlier work this paper cites.

Webshop: Towards scalable real-world web interaction with grounded language agents

Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022 · 2022

Earlier work this paper cites.

Mastering the game of no-press diplomacy via human-regularized reinforcement learning and planning

Anton Bakhtin, David J Wu, Adam Lerer, Jonathan Gray, Athul Paul Jacob, Gabriele Farina, Alexander H Miller, and Noam Brown. 2023 · 2023

Earlier work this paper cites.

Fireact: Toward language agent fine-tuning

Original

Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. 2023 · 2023

Earlier work this paper cites.

Plug-and-play policy planner for large language model powered dialogue agents

Yang Deng, Wenxuan Zhang, Wai Lam, See-Kiong Ng, and Tat-Seng Chua. 2023 · 2023

Earlier work this paper cites.

Improving language model negotiation with self-play and in-context learning from ai feedback

Original

Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. 2023 · 2023

Earlier work this paper cites.

Strategic reasoning with language models

Original

Kanishk Gandhi, Dorsa Sadigh, and Noah D Goodman. 2023 · 2023

Earlier work this paper cites.

Openagi: When llm meets domain experts

Yingqiang Ge, Wenyue Hua, Kai Mei, Juntao Tan, Shuyuan Xu, Zelong Li, Yongfeng Zhang, et al. 2023 · 2023

Earlier work this paper cites.

Reinforced self-training (rest) for language modeling

Original

Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. 2023 · 2023

Cited alongside, same era.

Reasoning with language model is planning with world model

Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. 2023 · 2023

Cited alongside, same era.

Mistral 7b

Original

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023 · 2023

Cited alongside, same era.

Self-refine: Iterative refinement with self-feedback

Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023 · 2023

Cited alongside, same era.

Training language models to self-correct via reinforcement learning

Original

Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. 2024 · 2024

Later among the works it cites.

Dialogue action tokens: Steering language models in goal-directed dialogue with a multi-turn planner

Original

Kenneth Li, Yiming Wang, Fernanda Viégas, and Martin Wattenberg. 2024 · 2024

Later among the works it cites.

Let’s verify step by step

Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024 · 2024

Later among the works it cites.

Large language models play starcraft II:benchmarks and a chain of summarization approach

Weiyu Ma, Qirui Mi, Yongcheng Zeng, Xue Yan, Runji Lin, Yuqiao Wu, Jun Wang, and Haifeng Zhang. 2024 · 2024

Later among the works it cites.

Introducing llama 3.1: Our most capable models to date

Meta. 2024 · 2024