Fetching the paper…

Reading the bibliography…

Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning · Around

2019

Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning

Hu, Hengyuan, Foerster, Jakob N

Original
PDF
Post on X

Understand

In recent years we have seen fast progress on a number of benchmark problems in AI, with modern methods achieving near or super human performance in Go, Poker and Dota.

One common aspect of all of these challenges is that they are by design adversarial or, technically speaking, zero-sum.
In contrast to these settings, success in the real world commonly requires humans to collaborate and communicate with others, in settings that are, at least partially, cooperative.
In the last year, the card game Hanabi has been established as a new benchmark environment for AI to fill this gap.

Built on

The Hanabi Challenge: A New Frontier for AI Research
Original
Nolan Bard, Jakob N. Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H. Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, Iain Dunning, Shibl Mourad, Hugo Larochelle, Marc G. Bellemare, and Michael Bowling · 1902
Earlier work this paper cites.
Learning to predict by the methods of temporal differences
Richard S. Sutton · 1988
Earlier work this paper cites.
Multi-agent reinforcement learning: Independent vs. cooperative agents
Ming Tan · 1993
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Hanabi simulation in rust
Jeff Wu · 1997
Earlier work this paper cites.
Deep Blue
Murray Campbell, A Joseph Hoane Jr, and Feng-hsiung Hsu · 2002
Earlier work this paper cites.
Decentralized pomdps
Frans A Oliehoek · 2012
Earlier work this paper cites.
Decentralized stochastic control with partial history sharing: A common information approach
Ashutosh Nayyar, Aditya Mahajan, and Demosthenis Teneketzis · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
Diederik P Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis · 2015
Earlier work this paper cites.
Prioritized Experience Replay
Original
Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver · 2015
Earlier work this paper cites.
Deep reinforcement learning with double q-learning
Original
Hado van Hasselt, Arthur Guez, and David Silver · 2015
Earlier work this paper cites.

Similar

Dueling network architectures for deep reinforcement learning
Original
Ziyu Wang, Nando de Freitas, and Marc Lanctot · 2015
Cited alongside, same era.
Learning to communicate with deep multi-agent reinforcement learning
Jakob Foerster, Ioannis Alexandros Assael, Nando de Freitas, and Shimon Whiteson · 2016
Cited alongside, same era.
Reinforcement learning with unsupervised auxiliary tasks
Original
Max Jaderberg, Volodymyr Mnih, Wojciech Marian Czarnecki, Tom Schaul, Joel Z Leibo, David Silver, and Koray Kavukcuoglu · 2016
Cited alongside, same era.
Learning to navigate in complex environments
Original
Piotr Mirowski, Razvan Pascanu, Fabio Viola, Hubert Soyer, Andrew J Ballard, Andrea Banino, Misha Denil, Ross Goroshin, Laurent Sifre, Koray Kavukcuoglu, et al · 2016
Cited alongside, same era.
Mastering the game of go with deep neural networks and tree search

Then

Distributed prioritized experience replay
Original
Dan Horgan, John Quan, David Budden, Gabriel Barth-Maron, Matteo Hessel, Hado van Hasselt, and David Silver · 2018
Later among the works it cites.
Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
Original
Max Jaderberg, Wojciech M. Czarnecki, Iain Dunning, Luke Marris, Guy Lever, Antonio Garcia Castaneda, Charles Beattie, Neil C. Rabinowitz, Ari S. Morcos, Avraham Ruderman, Nicolas Sonnerat, Tim Green, Louise Deason, Joel Z. Leibo, David Silver, Demis Hassabis, Koray Kavukcuoglu, and Thore Graepel · 2018
Later among the works it cites.
Deep reinforcement learning for multi-agent systems: a review of challenges, solutions and applications
Original
Thanh Thi Nguyen, Ngoc Duy Nguyen, and Saeid Nahavandi · 2018
Later among the works it cites.
QMIX: monotonic value function factorisation for deep multi-agent reinforcement learning
Original
Tabish Rashid, Mikayel Samvelyan, Christian Schröder de Witt, Gregory Farquhar, Jakob N. Foerster, and Shimon Whiteson · 2018

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al · 2016

Cited alongside, same era.

Learning Multiagent Communication with Backpropagation

Sainbayar Sukhbaatar, arthur szlam, and Rob Fergus · 2016

Cited alongside, same era.

Rational quantitative attribution of beliefs, desires and percepts in human mentalizing

Chris L Baker, Julian Jara-Ettinger, Rebecca Saxe, and Joshua B Tenenbaum · 2017

Cited alongside, same era.

Superhuman AI for heads-up no-limit poker: Libratus beats top professionals

Noam Brown and Tuomas Sandholm · 2017

Cited alongside, same era.

Multi-agent actor-critic for mixed cooperative-competitive environments

Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch · 2017

Cited alongside, same era.

Deepstack: Expert-level artificial intelligence in heads-up no-limit poker

Matej Moravčík, Martin Schmid, Neil Burch, Viliam Lisỳ, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, and Michael Bowling · 2017

Cited alongside, same era.

Mastering the game of go without human knowledge

David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al · 2017

Cited alongside, same era.

Value-decomposition networks for cooperative multi-agent learning

Original

Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinícius Flores Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z. Leibo, Karl Tuyls, and Thore Graepel · 2017

Cited alongside, same era.

Later among the works it cites.

A general reinforcement learning algorithm that masters chess, shogi, and go through self-play

David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al · 2018

Later among the works it cites.

Superhuman AI for multiplayer poker

Noam Brown and Tuomas Sandholm · 2019

Closest in time.

Diverse agents for ad-hoc cooperation in hanabi

Original

Rodrigo Canaan, Julian Togelius, Andy Nealen, and Stefan Menzel · 2019

Closest in time.

Bayesian action decoder for deep multi-agent reinforcement learning

Jakob Foerster, Francis Song, Edward Hughes, Neil Burch, Iain Dunning, Shimon Whiteson, Matthew Botvinick, and Michael Bowling · 2019

Closest in time.

Re-determinizing information set monte carlo tree search in hanabi

Original

James Goodman · 2019

Closest in time.

RECURRENT EXPERIENCE REPLAY IN DISTRIBUTED REINFORCEMENT LEARNING

Steven Kapturowski, Georg Ostrovski, John Quan, and Will Dabney · 2019

Closest in time.

Implicit communication of actionable information in human-ai teams

Claire Liang, Julia Proft, Erik Andersen, and Ross A Knepper · 2019

Closest in time.