Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccer

H. Kitano, M. Asada, Y. Kuniyoshi, I. Noda, and E. Osawa, “Robocup: The robot world cup initiative,” in Proceedings of the first international conference on Autonomous agents , 1997, pp. 340–347

1997

Earlier work this paper cites.

A. Merke and M. Riedmiller, “Karlsruhe brainstormers-a reinforcement learning approach to robotic soccer,” in RoboCup 2001: Robot Soccer World Cup V 5 . Springer, 2002, pp. 435–440

2002

Earlier work this paper cites.

P. Stone, R. S. Sutton, and G. Kuhlmann, “Reinforcement learning for RoboCup-soccer keepaway,” Adaptive Behavior , vol. 13, no. 3, pp. 165–188, 2005

2005

Earlier work this paper cites.

M. Riedmiller, T. Gabel, R. Hafner, and S. Lange, “Reinforcement learning for robot soccer,” Autonomous Robots , vol. 27, pp. 55–73, 2009

2009

Earlier work this paper cites.

R. Ros, J. L. Arcos, R. L. De Mantaras, and M. Veloso, “A case-based approach for coordinated action selection in robot soccer,” Artificial intelligence , vol. 173, no. 9-10, pp. 1014–1039, 2009

2009

Earlier work this paper cites.

D. Nardi, I. Noda, F. Ribeiro, P. Stone, O. von Stryk, and M. Veloso, “Robocup soccer leagues,” AI Magazine , vol. 35, no. 3, pp. 77–85, 2014

2014

Earlier work this paper cites.

M. Cutler, T. J. Walsh, and J. P. How, “Reinforcement learning with multi-fidelity simulators,” in 2014 IEEE International Conference on Robotics and Automation (ICRA) , 2014, pp. 3888–3895

2014

Earlier work this paper cites.

——, “Real-world reinforcement learning via multifidelity simulators,” IEEE Transactions on Robotics , vol. 31, no. 3, pp. 655–671, 2015

2015

Earlier work this paper cites.

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347 , 2017

Original

2017

Earlier work this paper cites.

I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas et al. , “Solving rubik’s cube with a robot hand,” arXiv preprint arXiv:1910.07113 , 2019

Original

2019

Earlier work this paper cites.

T. Li, H. Geyer, C. G. Atkeson, and A. Rai, “Using deep reinforcement learning to learn high-level policies on the atrias biped,” in 2019 International Conference on Robotics and Automation (ICRA) . IEEE, 2019, pp. 263–269

2019

Earlier work this paper cites.

O. Nachum, M. Ahn, H. Ponte, S. Gu, and V. Kumar, “Multi-agent manipulation via locomotion using hierarchical sim2real,” arXiv preprint arXiv:1908.05224 , 2019

Original

2019

Earlier work this paper cites.

S. Liu, G. Lever, J. Merel, S. Tunyasuvunakool, N. Heess, and T. Graepel, “Emergent coordination through competition,” arXiv preprint arXiv:1902.07151 , 2019

Original

2019

Earlier work this paper cites.

J. Siekmann, S. Valluri, J. Dao, L. Bermillo, H. Duan, A. Fern, and J. Hurst, “Learning memory-based control for human-scale bipedal locomotion,” arXiv preprint arXiv:2006.02402 , 2020

Original

2020

Earlier work this paper cites.

T. Li, N. Lambert, R. Calandra, F. Meier, and A. Rai, “Learning generalizable locomotion skills with hierarchical reinforcement learning,” in 2020 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2020, pp. 413–419

2020

Earlier work this paper cites.

V. Suryan, N. Gondhalekar, and P. Tokekar, “Multifidelity reinforcement learning with gaussian processes: model-based and model-free algorithms,” IEEE Robotics & Automation Magazine , vol. 27, no. 2, pp. 117–128, 2020

2020

Earlier work this paper cites.

Z. Li, X. Cheng, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath, “Reinforcement learning for robust parameterized locomotion control of bipedal robots,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 2811–2817

2021

Cited alongside, same era.

R. Beranek, M. Karimi, and M. Ahmadi, “A behavior-based reinforcement learning approach to control walking bipedal robots under unknown disturbances,” IEEE/ASME Transactions on Mechatronics , vol. 27, no. 5, pp. 2710–2720, 2021

2021

Cited alongside, same era.

C. Kouppas, M. Saada, Q. Meng, M. King, and D. Majoe, “Hybrid autonomous controller for bipedal robot balance with deep reinforcement learning and pattern generators,” Robotics and Autonomous Systems , vol. 146, p. 103891, 2021

2021

Cited alongside, same era.

T. Li, R. Calandra, D. Pathak, Y. Tian, F. Meier, and A. Rai, “Planning in learned latent action spaces for generalizable legged locomotion,” IEEE Robotics and Automation Letters , vol. 6, no. 2, pp. 2682–2689, 2021

A. Smit, H. A. Engelbrecht, W. Brink, and A. Pretorius, “Scaling multi-agent reinforcement learning to full 11 versus 11 simulated robotic football,” Autonomous Agents and Multi-Agent Systems , vol. 37, no. 1, p. 20, 2023

2023

Later among the works it cites.

M. Abreu, L. P. Reis, and N. Lau, “Designing a skilled soccer team for robocup: Exploring skill-set-primitives through reinforcement learning,” 2023. [Online]. Available: https://arxiv.org/abs/2312.14360

Original

2023

Later among the works it cites.

F. Lin, S. Huang, T. Pearce, W. Chen, and W.-W. Tu, “Tizero: Mastering multi-agent football with curriculum learning and self-play,” 2023. [Online]. Available: https://arxiv.org/abs/2302.07515

Original

2023

Later among the works it cites.

X. Huang, Z. Li, Y. Xiang, Y. Ni, Y. Chi, Y. Li, L. Yang, X. B. Peng, and K. Sreenath, “Creating a dynamic quadrupedal robotic goalkeeper with reinforcement learning,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2023, pp. 2715–2722