2017

Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions

Liu, Miao, Sivakumar, Kavinayan, Omidshafiei, Shayegan et al.

Understand

This paper presents a data-driven approach for multi-robot coordination in partially-observable domains based on Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) and macro-actions (MAs).

Dec-POMDPs provide a general framework for cooperative sequential decision making under uncertainty and MAs allow temporally extended and asynchronous action execution.
To date, most methods assume the underlying Dec-POMDP model is known a priori or a full simulator is available during planning time.
Previous methods which aim to address these issues suffer from local optimality and sensitivity to initial conditions.

Built on

J. Van den Berg, M. Lin, and D. Manocha, “Reciprocal velocity obstacles for real-time multi-agent navigation,” in
1935
Earlier work this paper cites.
L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and acting in partially observable stochastic domains,”
1998
Earlier work this paper cites.
R. S. Sutton, D. Precup, and S. Singh, “Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning,”
1999
Earlier work this paper cites.
D. Bernstein, S. Zilberstein, R. Washington, and J. Bresina, “Planetary rover control as a markov decision process,” in
2001
Earlier work this paper cites.
D. S. Bernstein, R. Givan, N. Immerman, and S. Zilberstein, “The complexity of decentralized control of Markov decision processes,”
2002
Earlier work this paper cites.
Y. Jin, A. A. Minai, and M. M. Polycarpou, “Cooperative real-time search and task allocation in UAV teams,” in
2003
Earlier work this paper cites.
D. Turra, L. Pollini, and M. Innocenti, “Fast unmanned vehicles task allocation with moving targets,” in
2004
Earlier work this paper cites.
K. Dresner and P. Stone, “Multiagent traffic management: Opportunities for multiagent learning,” in
2006
Earlier work this paper cites.
K. Jolly, K. Ravindran, R. Vijayakumar, and R. S. Kumar, “Intelligent decision making in multi-agent robot soccer system through compounded artificial neural networks,”
2007
Earlier work this paper cites.
H.-L. Choi, L. Brunet, and J. P. How, “Consensus-based decentralized auctions for robust task allocation,”
2009
Earlier work this paper cites.
M. Quigley, K. Conley, B. P. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng, “Ros: an open-source robot operating system,” in
2009
Earlier work this paper cites.

Similar

C. Amato, B. Bonet, and S. Zilberstein, “Finite-state controllers based on Mealy machines for centralized and decentralized POMDPs.” 2010
2010
Cited alongside, same era.
A. Kumar and S. Zilberstein, “Anytime planning for decentralized POMDPs using expectation maximization,” in
2010
Cited alongside, same era.
C. Y. Wong, G. Seet, and S. K. Sim, “Multiple-robot systems for USAR: key design attributes and deployment issues,”
2011
Cited alongside, same era.
D. Mellinger, N. Michael, and V. Kumar, “Trajectory generation and control for precise aggressive maneuvers with quadrotors,”
2012
Cited alongside, same era.
F. Wu, S. Zilberstein, and N. R. Jennings, “Monte-Carlo expectation maximization for decentralized POMDPs.” in
2013
Cited alongside, same era.
R. Horst and P. M. Pardalos,
2013
Cited alongside, same era.
D. Simon, “Evolutionary optimization algorithms: biologically-inspired and population-based approaches to computer intelligence. hoboken,” 2013
2013
Cited alongside, same era.
S. Grayson, “Search & Rescue using Multi-Robot Systems,”
2014
Cited alongside, same era.
C. Amato, G. D. Konidaris, and L. P. Kaelbling, “Planning with macro-actions in decentralized POMDPs,” in
2014
Cited alongside, same era.
C. Amato, G. Konidaris, G. Cruz, C. Maynor, J. How, and L. Kaelbling, “Planning for decentralized control of multiple robots under uncertainty,” in
2015
Cited alongside, same era.
C. Amato, G. Konidaris, A. Anders, G. Cruz, J. P. How, and L. P. Kaelbling, “Policy Search for Multi-Robot Coordination under Uncertainty,” in
2015
Cited alongside, same era.

Then

S. Omidshafiei, A. akbar Agha-mohammadi, C. Amato, and J. P. How, “Decentralized control of partially observable Markov decision processes using belief space macro-actions.” in
2015
Later among the works it cites.
A. Kumar, S. Zilberstein, and M. Toussaint, “Probabilistic inference techniques for scalable multiagent decision making,”
2015
Later among the works it cites.
M. Liu, C. Amato, X. Liao, J. P. How, and L. Carin, “Stick-Breaking Policy Learning in DEC-POMDPs,” in
2015
Later among the works it cites.
M. Gath,
2016
Later among the works it cites.
F. A. Oliehoek and C. Amato,
2016
Later among the works it cites.
S. Omidshafiei, A.-a. Agha-mohammadi, C. Amato, S.-Y. Liu, J. P. How, and J. Vian, “Graph-based cross entropy method for solving multi-robot decentralized pomdps,” in
2016
Later among the works it cites.
M. Liu, C. Amato, E. Anesta, J. Griffith, and J. How, “Learning for decentralized control of multiagent systems in large, partially-observable stochastic environments,” in
2016
Later among the works it cites.
Z. Song, X. Liao, and L. Carin, “Solving DEC-POMDPs by expectation maximization of value functions,” 2016
2016
Later among the works it cites.
D. Gossow and W. Woodall. (2016, nov) RVIZ
2016
Later among the works it cites.
S. Omidshafiei, A.-A. Agha-Mohammadi, Y. F. Chen, N. K. Ure, S.-Y. Liu, B. T. Lopez, R. Surati, J. P. How, and J. Vian, “Measurable augmented reality for prototyping cyberphysical systems: A robotics platform to aid the hardware prototyping and performance testing of algorithms,”
2016
Later among the works it cites.
L. Paull, J. Tani, H. Ahn, J. Alonso-Mora, L. Carlone, M. Cap, Y. F. Chen, C. Choi, J. Dusek, Y. Fang,
2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…