Fetching the paper…
Reading the bibliography…
This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots.
G. Feng, H. Zhang, Z. Li, X. B. Peng, B. Basireddy, L. Yue, Z. Song, L. Yang, Y. Liu, K. Sreenath et al. , “Genloco: Generalized locomotion controllers for quadrupedal robots,” in Conference on Robot Learning . PMLR, 2023, pp. 1893–1903
1903
Earlier work this paper cites.
M. H. Raibert, M. A. Chepponis, and H. B. Brown, “Experiments in balance with a 3d one-legged hopping machine,” International Journal of Robotics Research , vol. 3, no. 2, pp. 75 – 92, June 1984
1984
Earlier work this paper cites.
L. Ljung, “System identification,” in Signal analysis and prediction . Springer, 1998, pp. 163–173
1998
Earlier work this paper cites.
N. Meuleau, L. Peshkin, K.-E. Kim, and L. P. Kaelbling, “Learning finite-state controllers for partially observable environments,” in Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence , 1999, pp. 427–436
1999
Earlier work this paper cites.
S. Kajita, F. Kanehiro, K. Kaneko, K. Yokoi, and H. Hirukawa, “The 3d linear inverted pendulum mode: A simple modeling for a biped walking pattern generation,” in Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No. 01CH37180) , vol. 1. IEEE, 2001, pp. 239–246
2001
Earlier work this paper cites.
E. R. Westervelt, J. W. Grizzle, and D. E. Koditschek, “Hybrid zero dynamics of planar biped walkers,” IEEE transactions on automatic control , vol. 48, no. 1, pp. 42–56, 2003
2003
Earlier work this paper cites.
M. Vukobratović and B. Borovac, “Zero-moment point—thirty five years of its life,” International journal of humanoid robotics , vol. 1, no. 01, pp. 157–173, 2004
2004
Earlier work this paper cites.
L. Sentis and O. Khatib, “A whole-body control framework for humanoids operating in human environments,” in Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006. IEEE, 2006, pp. 2641–2648
2006
Earlier work this paper cites.
D. Goswami and P. Vadakkepat, “Planar bipedal jumping gaits with stable landing,” IEEE Transactions on Robotics , vol. 25, no. 5, pp. 1030–1046, 2009
2009
Earlier work this paper cites.
T. Takenaka, T. Matsumoto, T. Yoshiike, and S. Shirokura, “Real time motion generation and control for biped robot-2 nd report: Running gait pattern generation,” in 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, 2009, pp. 1092–1099
2009
Earlier work this paper cites.
T. Takenaka, T. Matsumoto, and T. Yoshiike, “Real time motion generation and control for biped robot-1 st report: Walking gait pattern generation,” in 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, 2009, pp. 1084–1091
2009
Earlier work this paper cites.
J. Rummel, Y. Blum, H. M. Maus, C. Rode, and A. Seyfarth, “Stable and robust walking with compliant legs,” in 2010 IEEE International Conference on Robotics and Automation . IEEE, 2010, pp. 5250–5255
2010
Earlier work this paper cites.
K. Bouyarmane and A. Kheddar, “Using a multi-objective controller to synthesize simulated humanoid robot motion with changing contact configurations,” in 2011 IEEE/RSJ international conference on intelligent robots and systems . IEEE, 2011, pp. 4414–4419
2011
Earlier work this paper cites.
K. Sreenath, H.-W. Park, I. Poulakakis, and J. W. Grizzle, “A compliant hybrid zero dynamics controller for stable, efficient and fast bipedal walking on mabel,” The International Journal of Robotics Research , vol. 30, no. 9, pp. 1170–1193, 2011
2011
Earlier work this paper cites.
I. D. Landau, R. Lozano, M. M’Saad, and A. Karimi, Adaptive control: algorithms, analysis and applications . Springer Science & Business Media, 2011
2011
Earlier work this paper cites.
J. Pratt, T. Koolen, T. De Boer, J. Rebula, S. Cotton, J. Carff, M. Johnson, and P. Neuhaus, “Capturability-based analysis and control of legged locomotion, part 2: Application to m2v2, a lower-body humanoid,” The international journal of robotics research , vol. 31, no. 10, pp. 1117–1133, 2012
2012
Earlier work this paper cites.
M. T. Spaan, “Partially observable markov decision processes,” in Reinforcement learning: State-of-the-art . Springer, 2012, pp. 387–414
2012
Earlier work this paper cites.
E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A physics engine for model-based control,” in 2012 IEEE/RSJ international conference on intelligent robots and systems , 2012, pp. 5026–5033
2012
Earlier work this paper cites.
D. E. Orin, A. Goswami, and S.-H. Lee, “Centroidal dynamics of a humanoid robot,” Autonomous robots , vol. 35, pp. 161–176, 2013
2013
Earlier work this paper cites.
K. Sreenath, H.-W. Park, I. Poulakakis, and J. W. Grizzle, “Embedding active force control within the compliant hybrid zero dynamics to achieve stable, fast running on mabel,” The International Journal of Robotics Research , vol. 32, no. 3, pp. 324–345, 2013
2013
Earlier work this paper cites.
S. Kuindersma, F. Permenter, and R. Tedrake, “An efficiently solvable quadratic program for stabilizing dynamic locomotion,” in 2014 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2014, pp. 2589–2594
2014
Earlier work this paper cites.
M. Posa, C. Cantu, and R. Tedrake, “A direct method for trajectory optimization of rigid bodies through contact,” The International Journal of Robotics Research , vol. 33, no. 1, pp. 69–81, 2014
2014
Earlier work this paper cites.
R. Deits and R. Tedrake, “Footstep planning on uneven terrain with mixed-integer convex optimization,” in 2014 IEEE-RAS international conference on humanoid robots . IEEE, 2014, pp. 279–286
2014
Earlier work this paper cites.
A. Ibanez, P. Bidaud, and V. Padois, “Emergence of humanoid walking behaviors from mixed-integer model predictive control,” in 2014 IEEE/RSJ international conference on intelligent robots and systems . IEEE, 2014, pp. 4014–4021
2014
Earlier work this paper cites.
H. Dai, A. Valenzuela, and R. Tedrake, “Whole-body motion planning with centroidal dynamics and full kinematics,” in 2014 IEEE-RAS International Conference on Humanoid Robots . IEEE, 2014, pp. 295–302
2014
Earlier work this paper cites.
T. Marcucci, M. Gabiccini, and A. Artoni, “A two-stage trajectory optimization strategy for articulated bodies with unscheduled contact sequences,” IEEE Robotics and Automation Letters , vol. 2, no. 1, pp. 104–111, 2016
2016
Earlier work this paper cites.
P. M. Wensing and D. E. Orin, “Improved computation of the humanoid centroidal dynamics and application for whole-body control,” International Journal of Humanoid Robotics , vol. 13, no. 01, p. 1550039, 2016
2016
Earlier work this paper cites.
X. Da, O. Harib, R. Hartley, B. Griffin, and J. W. Grizzle, “From 2d design of underactuated bipedal gaits to 3d implementation: Walking with speed tracking,” IEEE Access , vol. 4, pp. 3469–3478, 2016
2016
Earlier work this paper cites.
W.-L. Ma, S. Kolathaya, E. R. Ambrose, C. M. Hubicki, and A. D. Ames, “Bipedal robotic running with durus-2d: Bridging the gap between theory and experiment,” in Proceedings of the 20th international conference on hybrid systems: computation and control , 2017, pp. 265–274
2017
Earlier work this paper cites.
2017
Earlier work this paper cites.
X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim-to-real transfer of robotic control with dynamics randomization,” in 2018 IEEE international conference on robotics and automation (ICRA) , 2018, pp. 3803–3810
2018
Earlier work this paper cites.
X. Xiong and A. D. Ames, “Bipedal hopping: Reduced-order model embedding via optimization-based control,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2018, pp. 3821–3828
2018
Earlier work this paper cites.
A. Hereid, C. M. Hubicki, E. A. Cousineau, and A. D. Ames, “Dynamic humanoid locomotion: A scalable formulation for hzd gait optimization,” IEEE Transactions on Robotics , vol. 34, no. 2, pp. 370–387, 2018
2018
Earlier work this paper cites.
Z. Xie, G. Berseth, P. Clary, J. Hurst, and M. van de Panne, “Feedback control for cassie with deep reinforcement learning,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2018, pp. 1241–1246
2018
Earlier work this paper cites.
SFU, “Sfu motion capture database,” https://mocap.cs.sfu.ca/
2018
Earlier work this paper cites.
Y. Lee, S.-H. Sun, S. Somasundaram, E. S. Hu, and J. J. Lim, “Composing complex skills by learning transition policies,” in International Conference on Learning Representations , 2018
2018
Earlier work this paper cites.
Y. Gong, R. Hartley, X. Da, A. Hereid, O. Harib, J.-K. Huang, and J. Grizzle, “Feedback control of a cassie bipedal robot: Walking, standing, and riding a segway,” in 2019 American Control Conference (ACC) . IEEE, 2019, pp. 4559–4566
2019
Earlier work this paper cites.
A. Hereid, O. Harib, R. Hartley, Y. Gong, and J. W. Grizzle, “Rapid trajectory optimization using c-frost with illustration on a cassie-series dynamic walking biped,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2019, pp. 4722–4729
2019
Cited alongside, same era.
F. L. Moro and L. Sentis, “Whole-body control of humanoid robots,” Humanoid Robotics: A reference, Springer, Dordrecht , 2019
2019
Cited alongside, same era.
S. Caron, A. Kheddar, and O. Tempier, “Stair climbing stabilization of the hrp-4 humanoid robot using whole-body admittance control,” in 2019 International conference on robotics and automation (ICRA) . IEEE, 2019, pp. 277–283
2019
Cited alongside, same era.
K. Kojima, Y. Kojio, T. Ishikawa, F. Sugai, Y. Kakiuchi, K. Okada, and M. Inaba, “A robot design method for weight saving aimed at dynamic motions: Design of humanoid jaxon3-p and realization of jump motions,” in 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids) . IEEE, 2019, pp. 586–593
B. Landry, J. Lorenzetti, Z. Manchester, and M. Pavone, “Bilevel optimization for planning through contact: A semidirect method,” in Robotics Research: The 19th International Symposium ISRR , 2022, pp. 789–804
2022
Later among the works it cites.
R. Deits, S. Kuindersma, M. P. Kelly, T. Koolen, Y. Abe, and B. Stephens, “Robot movement and online trajectory optimization,” Dec. 29 2022, uS Patent App. 17/358,628
2022
Later among the works it cites.
G. Margolis, G. Yang, K. Paigwar, T. Chen, and P. Agrawal, “Rapid locomotion via reinforcement learning,” in Robotics: Science and Systems , 2022
2022
Later among the works it cites.
A. Kumar, Z. Li, J. Zeng, D. Pathak, K. Sreenath, and J. Malik, “Adapting rapid motor adaptation for bipedal robots,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2022, pp. 1161–1168
2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2019
Cited alongside, same era.
J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots,” Science Robotics , vol. 4, no. 26, p. eaau5872, 2019
2019
Cited alongside, same era.
T. Haarnoja, S. Ha, A. Zhou, J. Tan, G. Tucker, and S. Levine, “Learning to walk via deep reinforcement learning,” Robotics: Science and Systems (RSS) , 2019
2019
Cited alongside, same era.
W. Yu, V. C. Kumar, G. Turk, and C. K. Liu, “Sim-to-real transfer for biped locomotion,” in 2019 ieee/rsj international conference on intelligent robots and systems (iros) , 2019, pp. 3503–3510
2019
Cited alongside, same era.
Z. Li, C. Cummings, and K. Sreenath, “Animated cassie: A dynamic relatable robotic character,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2020, pp. 3739–3746
2020
Cited alongside, same era.
Z. Xie, P. Clary, J. Dao, P. Morais, J. Hurst, and M. Panne, “Learning locomotion skills for cassie: Iterative design and sim-to-real,” in Conference on Robot Learning . PMLR, 2020, pp. 317–329
2020
Cited alongside, same era.
J. Siekmann, S. Valluri, J. Dao, L. Bermillo, H. Duan, A. Fern, and J. Hurst, “Learning memory-based control for human-scale bipedal locomotion,” in Robotics science and systems , 2020
2020
Cited alongside, same era.
M. Fevre, P. M. Wensing, and J. P. Schmiedeler, “Rapid bipedal gait optimization in casadi,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2020, pp. 3672–3678
2020
Cited alongside, same era.
P. Fernbach, S. Tonneau, O. Stasse, J. Carpentier, and M. Taïx, “C-croc: Continuous and convex resolution of centroidal dynamic trajectories for legged robots in multicontact scenarios,” IEEE Transactions on Robotics , vol. 36, no. 3, pp. 676–691, 2020
2020
Cited alongside, same era.
A. Escontrela, X. B. Peng, W. Yu, T. Zhang, A. Iscen, K. Goldberg, and P. Abbeel, “Adversarial motion priors make good substitutes for complex reward functions,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2022, pp. 25–32
2022
Later among the works it cites.
T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter, “Learning robust perceptive locomotion for quadrupedal robots in the wild,” Science Robotics , vol. 7, no. 62, p. eabk2822, 2022
2022
Later among the works it cites.
G. A. Castillo, B. Weng, W. Zhang, and A. Hereid, “Reinforcement learning-based cascade motion policy design for robust 3d bipedal locomotion,” IEEE Access , vol. 10, pp. 20 135–20 148, 2022
2022
Later among the works it cites.
L. Smith, J. C. Kew, X. B. Peng, S. Ha, J. Tan, and S. Levine, “Legged robots that keep on learning: Fine-tuning locomotion policies in the real world,” in 2022 International Conference on Robotics and Automation (ICRA) . IEEE, 2022, pp. 1593–1599
2022
Later among the works it cites.
T. Westenbroek, F. Castaneda, A. Agrawal, S. Sastry, and K. Sreenath, “Lyapunov design for robust and efficient robotic reinforcement learning,” in 6th Annual Conference on Robot Learning , 2022
2022
Later among the works it cites.
G. Ji, J. Mun, H. Kim, and J. Hwangbo, “Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion,” IEEE Robotics and Automation Letters , vol. 7, no. 2, pp. 4630–4637, 2022
2022
Later among the works it cites.
M. Bogdanovic, M. Khadiv, and L. Righetti, “Model-free reinforcement learning for robust locomotion using demonstrations from trajectory optimization,” Frontiers in Robotics and AI , vol. 9, p. 854212, 2022
2022
Later among the works it cites.
A. M. Annaswamy, “Adaptive control and intersections with reinforcement learning,” Annual Review of Control, Robotics, and Autonomous Systems , vol. 6, pp. 65–93, 2023
2023
Later among the works it cites.
Z. Li, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath, “Robust and versatile bipedal jumping control through reinforcement learning,” Robotics: Science and Systems XIX, Daegu, Republic of Korea , 2023
2023
Later among the works it cites.
2023
Later among the works it cites.
D. Crowley, J. Dao, H. Duan, K. Green, J. Hurst, and A. Fern, “Optimizing bipedal locomotion for the 100m dash with comparison to human running,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 12 205–12 211
2023
Later among the works it cites.
P. M. Wensing, M. Posa, Y. Hu, A. Escande, N. Mansard, and A. Del Prete, “Optimization-based control for dynamic legged robots,” IEEE Transactions on Robotics , 2023
2023
Later among the works it cites.
H. Qi, X. Chen, Z. Yu, G. Huang, Y. Liu, L. Meng, and Q. Huang, “Vertical jump of a humanoid robot with cop-guided angular momentum control and impact absorption,” IEEE Transactions on Robotics , 2023
2023
Later among the works it cites.
A. Meduri, P. Shah, J. Viereck, M. Khadiv, I. Havoutis, and L. Righetti, “Biconmp: A nonlinear model predictive control framework for whole body motion planning,” IEEE Transactions on Robotics , vol. 39, no. 2, pp. 905–922, 2023
2023
Later among the works it cites.
S. Chen, B. Zhang, M. W. Mueller, A. Rai, and K. Sreenath, “Learning torque control for quadrupedal locomotion,” in 2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids) , 2023, pp. 1–8
2023
Later among the works it cites.
Z. Fu, X. Cheng, and D. Pathak, “Deep whole-body control: learning a unified policy for manipulation and locomotion,” in Conference on Robot Learning . PMLR, 2023, pp. 138–149
2023
Later among the works it cites.
X. Huang, Z. Li, Y. Xiang, Y. Ni, Y. Chi, Y. Li, L. Yang, X. B. Peng, and K. Sreenath, “Creating a dynamic quadrupedal robotic goalkeeper with reinforcement learning,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2023, pp. 2715–2722
2023
Later among the works it cites.
R. P. Singh, Z. Xie, P. Gergondet, and F. Kanehiro, “Learning bipedal walking for humanoids with current feedback,” IEEE Access , 2023
2023
Later among the works it cites.
P. Wu, A. Escontrela, D. Hafner, P. Abbeel, and K. Goldberg, “Daydreamer: World models for physical robot learning,” in Conference on Robot Learning . PMLR, 2023, pp. 2226–2240
2023
Later among the works it cites.
L. Smith, I. Kostrikov, and S. Levine, “Demonstrating a walk in the park: Learning to walk in 20 minutes with model-free reinforcement learning,” Robotics: Science and Systems (RSS) Demo , vol. 2, no. 3, p. 4, 2023
2023
Later among the works it cites.
G. B. Margolis and P. Agrawal, “Walk these ways: Tuning robot control for generalization with multiplicity of behavior,” in Conference on Robot Learning . PMLR, 2023, pp. 22–31
2023
Later among the works it cites.
X. Cheng, A. Kumar, and D. Pathak, “Legs as manipulator: Pushing quadrupedal agility beyond locomotion,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) , 2023, pp. 5106–5112
2023
Later among the works it cites.
2023
Later among the works it cites.
2023
Later among the works it cites.
D. Lim, M.-J. Kim, J. Cha, D. Kim, and J. Park, “Proprioceptive external torque learning for floating base robot and its applications to humanoid locomotion,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2023
2023
Later among the works it cites.
DRL, “cassie-mujoco-sim,” 2023. [Online]. Available: https://github.com/osudrl/cassie-mujoco-sim
2023
Later among the works it cites.
D. Kim, G. Berseth, M. Schwartz, and J. Park, “Torque-based deep reinforcement learning for task-and-robot agnostic learning on bipedal robots using sim-to-real transfer,” IEEE Robotics and Automation Letters , 2023
2023
Later among the works it cites.
S. Liu, M. Xu, P. Huang, X. Zhang, Y. Liu, K. Oguchi, and D. Zhao, “Continual vision-based reinforcement learning with group symmetries,” in Conference on Robot Learning . PMLR, 2023, pp. 222–240
2023
Later among the works it cites.
S. Le Cleac’h, T. A. Howell, S. Yang, C.-Y. Lee, J. Zhang, A. Bishop, M. Schwager, and Z. Manchester, “Fast contact-implicit model predictive control,” IEEE Transactions on Robotics , 2024
2024
Closest in time.
I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath, “Real-world humanoid locomotion with reinforcement learning,” Science Robotics , vol. 9, no. 89, p. eadi9579, 2024
2024
Closest in time.
2024
Closest in time.