2018

Deep Multi-Agent Reinforcement Learning with Relevance Graphs

Malysheva, Aleksandra, Sung, Tegg Taekyong, Sohn, Chae-Bong et al.

Understand

Over recent years, deep reinforcement learning has shown strong successes in complex single-agent tasks, and more recently this approach has also been applied to multi-agent domains.

In this paper, we propose a novel approach, called MAGnet, to multi-agent reinforcement learning (MARL) that utilizes a relevance graph representation of the environment obtained by a self-attention mechanism, and a message-generation technique inspired by the NerveNet architecture.
We applied our MAGnet approach to the Pommerman game and the results show that it significantly outperforms state-of-the-art MARL solutions, including DQN, MADDPG, and MCTS.

Built on

Multi-agent reinforcement learning: Independent vs. cooperative agents
M. Tan · 1993
Earlier work this paper cites.
Policy gradient methods for reinforcement learning with function approximation
R. S. Sutton, D. A. McAllester, S. P. Singh, and Y. Mansour · 2000
Earlier work this paper cites.
Distributed computing: fundamentals, simulations, and advanced topics
H. Attiya and J. Welch · 2004
Earlier work this paper cites.
Extending q-learning to general adaptive multi-agent systems
G. Tesauro · 2004
Earlier work this paper cites.
Dynamic programming and optimal control
D. P. Bertsekas, D. P. Bertsekas, D. P. Bertsekas, and D. P. Bertsekas · 2005
Earlier work this paper cites.
Monte-carlo tree search: A new framework for game ai
G. Chaslot, S. Bakkes, I. Szita, and P. Spronck · 2008
Earlier work this paper cites.

Similar

Independent reinforcement learners in cooperative markov games: a survey regarding coordination problems
L. Matignon, G. J. Laurent, and N. Le Fort-Piat · 2012
Cited alongside, same era.
Playing atari with deep reinforcement learning
Original
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller · 2013
Cited alongside, same era.
Markov decision processes: discrete stochastic dynamic programming
M. L. Puterman · 2014
Cited alongside, same era.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Original
S. Ioffe and C. Szegedy · 2015
Cited alongside, same era.
Continuous control with deep reinforcement learning
Original
T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra · 2015
Cited alongside, same era.
Multi-agent deep reinforcement learning, 2016
M. Egorov · 2016
Cited alongside, same era.

Then

Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Later among the works it cites.
Multi-agent actor-critic for mixed cooperative-competitive environments
R. Lowe, Y. Wu, A. Tamar, J. Harb, O. P. Abbeel, and I. Mordatch · 2017
Later among the works it cites.
Attention is all you need
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin · 2017
Later among the works it cites.
Learning to search with mctsnets
Original
A. Guez, T. Weber, I. Antonoglou, K. Simonyan, O. Vinyals, D. Wierstra, R. Munos, and D. Silver · 2018
Closest in time.
Pommerman baselines
T. Matiisen · 2018
Closest in time.
Nervenet: Learning structured policy with graph neural networks
T. Wang, R. Liao, J. Ba, and S. Fidler · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…