2018

Efficient Collaborative Multi-Agent Deep Reinforcement Learning for Large-Scale Fleet Management

Lin, Kaixiang, Zhao, Renyu, Xu, Zhe et al.

Understand

Large-scale online ride-sharing platforms have substantially transformed our lives by reallocating transportation resources to alleviate traffic congestion and promote transportation efficiency.

An efficient fleet management strategy not only can significantly improve the utilization of transportation resources but also increase the revenue and customer satisfaction.
It is a challenging task to design an effective fleet management strategy that can adapt to an environment involving complex dynamics between demand and supply.
Existing studies usually work on a simplified problem setting that can hardly capture the complicated stochastic demand-supply variations in high-dimensional space.

Built on

V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” in ICML , 2016, pp. 1928–1937
1937
Earlier work this paper cites.
P. J. Dejax and T. G. Crainic, “Survey paper—a review of empty flows and fleet management models in freight transportation,” Transportation science , vol. 21, no. 4, pp. 227–248, 1987
1987
Earlier work this paper cites.
M. Tan, “Multi-agent reinforcement learning: Independent vs. cooperative agents,” in ICML , 1993, pp. 330–337
1993
Earlier work this paper cites.
R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction . MIT press Cambridge, 1998, vol. 1, no. 1
1998
Earlier work this paper cites.
G. A. Godfrey and W. B. Powell, “An adaptive dynamic programming algorithm for dynamic fleet management, i: Single period travel times,” Transportation Science , vol. 36, no. 1, pp. 21–39, 2002
2002
Earlier work this paper cites.
——, “An adaptive dynamic programming algorithm for dynamic fleet management, ii: Multiperiod travel times,” Transportation Science , vol. 36, no. 1, pp. 40–54, 2002
2002
Earlier work this paper cites.
L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent reinforcement learning,” IEEE Transactions on Systems, Man, And Cybernetics-Part C: Applications and Reviews, 38 (2), 2008 , 2008
2008
Earlier work this paper cites.
B. Bakker, S. Whiteson, L. Kester, and F. C. Groen, “Traffic light control by multiagent reinforcement learning systems,” in Interactive Collaborative Information Systems . Springer, 2010, pp. 475–510
2010
Earlier work this paper cites.
K. T. Seow, N. H. Dang, and D.-H. Lee, “A collaborative multiagent taxi-dispatch system,” IEEE T-ASE , vol. 7, no. 3, pp. 607–616, 2010
2010
Earlier work this paper cites.
M. Maciejewski and K. Nagel, “The influence of multi-agent cooperation on the efficiency of taxi dispatching,” in PPAM . Springer, 2013, pp. 751–760
2013
Earlier work this paper cites.
M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling, “The arcade learning environment: An evaluation platform for general agents.” J. Artif. Intell. Res.(JAIR) , vol. 47, pp. 253–279, 2013
2013
Earlier work this paper cites.

Similar

K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” in ECCV . Springer, 2016, pp. 630–645
2014
Cited alongside, same era.
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al. , “Human-level control through deep reinforcement learning,” Nature , vol. 518, no. 7540, pp. 529–533, 2015
2015
Cited alongside, same era.
D.-A. Clevert, T. Unterthiner, and S. Hochreiter, “Fast and accurate deep network learning by exponential linear units (elus),” arXiv preprint arXiv:1511.07289 , 2015
Original
2015
Cited alongside, same era.
Z. Li, Y. Hong, and Z. Zhang, “Do on-demand ride-sharing services affect traffic congestion? evidence from uber entry,” Working paper, available at SSRN: https://ssrn. com/abstract= 2838043, Tech. Rep., 2016
2016
Cited alongside, same era.
D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot et al. , “Mastering the game of go with deep neural networks and tree search,” Nature , vol. 529, no. 7587, pp. 484–489, 2016
2016
Cited alongside, same era.
G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” arXiv preprint arXiv:1606.01540 , 2016
Original
2016
Cited alongside, same era.
D. Bahdanau, P. Brakel, K. Xu, A. Goyal, R. Lowe, J. Pineau, A. Courville, and Y. Bengio, “An actor-critic algorithm for sequence prediction,” arXiv preprint arXiv:1607.07086 , 2016
Original
2016
Cited alongside, same era.
C. Wei, Y. Wang, X. Yan, and C. Shao, “Look-ahead insertion policy for a shared-taxi system based on reinforcement learning,” IEEE Access , 2017
2017
Cited alongside, same era.
D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton et al. , “Mastering the game of go without human knowledge,” Nature , vol. 550, no. 7676, p. 354, 2017
2017
Cited alongside, same era.
K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, “A brief survey of deep reinforcement learning,” arXiv preprint arXiv:1708.05866 , 2017
Original
2017
Cited alongside, same era.
T.-H. Pham, G. De Magistris, and R. Tachibana, “Optlayer-practical constrained optimization for deep reinforcement learning in the real world,” arXiv preprint arXiv:1709.07643 , 2017
Original
2017
Cited alongside, same era.
Uber. [Online]. Available: https://www.uber.com/
Cited in the paper.

Then

R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” arXiv preprint arXiv:1706.02275 , 2017
Original
2017
Later among the works it cites.
J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, “Counterfactual multi-agent policy gradients,” arXiv preprint arXiv:1705.08926 , 2017
Original
2017
Later among the works it cites.
A. Tampuu, T. Matiisen, D. Kodelja, I. Kuzovkin, K. Korjus, J. Aru, J. Aru, and R. Vicente, “Multiagent cooperation and competition with deep reinforcement learning,” PloS one , vol. 12, no. 4, p. e0172395, 2017
2017
Later among the works it cites.
L. Zheng, J. Yang, H. Cai, W. Zhang, J. Wang, and Y. Yu, “Magent: A many-agent reinforcement learning platform for artificial collective intelligence,” arXiv preprint arXiv:1712.00600 , 2017
Original
2017
Later among the works it cites.
D. T. Nguyen, A. Kumar, and H. C. Lau, “Collective multiagent sequential decision making under uncertainty,” AAAI , 2017
2017
Later among the works it cites.
——, “Policy gradient with value function approximation for collective multiagent planning,” NIPS , 2017
2017
Later among the works it cites.
P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V. Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuyls et al. , “Value-decomposition networks for cooperative multi-agent learning,” arXiv preprint arXiv:1706.05296 , 2017
Original
2017
Later among the works it cites.
Y. Yang, R. Luo, M. Li, M. Zhou, W. Zhang, and J. Wang, “Mean field multi-agent reinforcement learning,” ICML , 2018
2018
Closest in time.
T. Rashid, M. Samvelyan, C. S. de Witt, G. Farquhar, J. Foerster, and S. Whiteson, “Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning,” arXiv preprint arXiv:1803.11485 , 2018
Original
2018
Closest in time.
J. S. Dibangoye and O. Buffet, “Learning to act in decentralized partially observable mdps,” Ph.D. dissertation, INRIA Grenoble-Rhone-Alpes-CHROMA Team; INRIA Nancy, équipe LARSEN, 2018
2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…