HERMES: Human-to-Robot Embodied Learning from Multi-Source Motion Data for Mobile Dexterous Manipulation

M. J. Willis, “Proportional-integral-derivative control,” Dept. of Chemical and Process Engineering University of Newcastle , vol. 6, 1999

1999

Earlier work this paper cites.

G. Bradski, A. Kaehler et al. , “Opencv,” Dr. Dobb’s journal of software tools , vol. 3, no. 2, 2000

2000

Earlier work this paper cites.

K. Madsen, H. B. Nielsen, and O. Tingleff, “Methods for non-linear least squares problems,” Informatics and Mathematical Modelling Technical University of Denmark , vol. 1, 2004

2004

Earlier work this paper cites.

B. D. Argall, S. Chernova, M. Veloso, and B. Browning, “A survey of robot learning from demonstration,” Robotics and autonomous systems , vol. 57, no. 5, pp. 469–483, 2009

2009

Earlier work this paper cites.

M. Zuliani, “Ransac for dummies,” Vision Research Lab, University of California, Santa Barbara , vol. 1, 2009

2009

Earlier work this paper cites.

S. Yenamandra, A. Ramachandran, K. Yadav, A. S. Wang, M. Khanna, T. Gervet, T.-Y. Yang, V. Jain, A. Clegg, J. M. Turner et al. , “Homerobot: Open-vocabulary mobile manipulation,” in Conference on Robot Learning . PMLR, 2023, pp. 1975–2011

2011

Earlier work this paper cites.

E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A physics engine for model-based control,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, 2012, pp. 5026–5033

2012

Earlier work this paper cites.

S. Li, C. Xu, and M. Xie, “A robust o (n) solution to the perspective-n-point problem,” IEEE transactions on pattern analysis and machine intelligence , vol. 34, no. 7, pp. 1444–1450, 2012

2012

Earlier work this paper cites.

N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “Indoor segmentation and support inference from rgbd images,” in Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part V 12 . Springer, 2012, pp. 746–760

2012

Earlier work this paper cites.

E. Eade, “Gauss-newton/levenberg-marquardt optimization,” Tech. Rep. , 2013

2013

Earlier work this paper cites.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

2016

Earlier work this paper cites.

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347 , 2017

Original

2017

Earlier work this paper cites.

X. B. Peng, P. Abbeel, S. Levine, and M. Van de Panne, “Deepmimic: Example-guided deep reinforcement learning of physics-based character skills,” ACM Transactions On Graphics (TOG) , vol. 37, no. 4, pp. 1–14, 2018

2018

Earlier work this paper cites.

Y. Wu and K. He, “Group normalization,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 3–19

2018

Earlier work this paper cites.

M. Labbé and F. Michaud, “Rtab‐map as an open‐source lidar and visual simultaneous localization and mapping library for large‐scale and long‐term online operation,” Journal of Field Robotics , vol. 36, no. 2, p. 416–446, Oct. 2018. [Online]. Available: http://dx.doi.org/10.1002/rob.21831

2018

Earlier work this paper cites.

Z. Shan, R. Li, and S. Schwertfeger, “Rgbd-inertial trajectory estimation and mapping for ground robots,” Sensors , vol. 19, no. 10, 2019. [Online]. Available: https://www.mdpi.com/1424-8220/19/10/2251

2019

Earlier work this paper cites.

A. Handa, K. Van Wyk, W. Yang, J. Liang, Y.-W. Chao, Q. Wan, S. Birchfield, N. Ratliff, and D. Fox, “Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system,” in 2020 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2020, pp. 9164–9170

2020

Earlier work this paper cites.

C. Campos, R. Elvira, J. J. G. Rodriguez, J. M. M. Montiel, and J. D. Tardos, “Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam,” IEEE Transactions on Robotics , vol. 37, no. 6, p. 1874–1890, Dec. 2021. [Online]. Available: http://dx.doi.org/10.1109/TRO.2021.3075644

2021

Earlier work this paper cites.

P. Mandikal and K. Grauman, “Dexvip: Learning dexterous grasping with human hand pose priors from video,” in Conference on Robot Learning . PMLR, 2022, pp. 651–661

2022

Earlier work this paper cites.

Y. Qin, Y.-H. Wu, S. Liu, H. Jiang, R. Yang, Y. Fu, and X. Wang, “Dexmv: Imitation learning for dexterous manipulation from human videos,” in European Conference on Computer Vision . Springer, 2022, pp. 570–587

2022

Earlier work this paper cites.

Z. Yuan, Z. Xue, B. Yuan, X. Wang, Y. Wu, Y. Gao, and H. Xu, “Pre-trained image encoder for generalizable visual reinforcement learning,” Advances in Neural Information Processing Systems , vol. 35, pp. 13 022–13 037, 2022

2022

Earlier work this paper cites.

N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” in Proceedings of the 5th Conference on Robot Learning , ser. Proceedings of Machine Learning Research, vol. 164. PMLR, 2022, pp. 91–100. [Online]. Available: https://proceedings.mlr.press/v164/rudin22a.html

2022

Earlier work this paper cites.

C. Wang, L. Fan, J. Sun, R. Zhang, L. Fei-Fei, D. Xu, Y. Zhu, and A. Anandkumar, “Mimicplay: Long-horizon imitation learning by watching human play,” in 7th Annual Conference on Robot Learning , 2023

2023

Earlier work this paper cites.

Y. Qin, W. Yang, B. Huang, K. Van Wyk, H. Su, X. Wang, Y.-W. Chao, and D. Fox, “Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system,” arXiv preprint arXiv:2307.04577 , 2023

Original

2023

Earlier work this paper cites.

K. Shaw, S. Bahl, and D. Pathak, “Videodex: Learning dexterity from internet videos,” in Conference on Robot Learning . PMLR, 2023, pp. 654–665

2023

Earlier work this paper cites.

Y. Chen, C. Wang, L. Fei-Fei, and K. Liu, “Sequential dexterity: Chaining dexterous policies for long-horizon manipulation,” in Conference on Robot Learning . PMLR, 2023, pp. 3809–3829

2023

Earlier work this paper cites.

D. Shah, A. Sridhar, N. Dashora, K. Stachowicz, K. Black, N. Hirose, and S. Levine, “Vint: A foundation model for visual navigation,” in Conference on Robot Learning . PMLR, 2023, pp. 711–733

2023

Earlier work this paper cites.

J. Ye, J. Wang, B. Huang, Y. Qin, and X. Wang, “Learning continuous grasping function with a dexterous hand from human demonstrations,” IEEE Robotics and Automation Letters , vol. 8, no. 5, pp. 2882–2889, 2023

2023

Earlier work this paper cites.

Z. Zhuang, Z. Fu, J. Wang, C. G. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao, “Robot parkour learning,” in Conference on Robot Learning . PMLR, 2023, pp. 73–92

2023

Earlier work this paper cites.

H. Qi, A. Kumar, R. Calandra, Y. Ma, and J. Malik, “In-hand object rotation via rapid motor adaptation,” in Conference on Robot Learning . PMLR, 2023, pp. 1722–1732

2023

Earlier work this paper cites.

L. Wang, Y. Ling, Z. Yuan, M. Shridhar, C. Bao, Y. Qin, B. Wang, H. Xu, and X. Wang, “Gensim: Generating robotic simulation tasks via large language models,” in The Twelfth International Conference on Learning Representations , 2023

2023

Earlier work this paper cites.

B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering.” ACM Trans. Graph. , vol. 42, no. 4, pp. 139–1, 2023

2023

Earlier work this paper cites.

X. Huang, D. Batra, A. Rai, and A. Szot, “Skill transformer: A monolithic policy for mobile manipulation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 10 852–10 862

2023

Cited alongside, same era.

J. Wu, R. Antonova, A. Kan, M. Lepert, A. Zeng, S. Song, J. Bohg, S. Rusinkiewicz, and T. Funkhouser, “Tidybot: Personalized robot assistance with large language models,” Autonomous Robots , vol. 47, no. 8, pp. 1087–1102, 2023

2023

Cited alongside, same era.

H.-S. Fang, C. Wang, H. Fang, M. Gou, J. Liu, H. Yan, W. Liu, Y. Xie, and C. Lu, “Anygrasp: Robust and efficient grasp perception in spatial and temporal domains,” IEEE Transactions on Robotics (T-RO) , 2023

2023

Cited alongside, same era.

L. Medeiros, “Lang segment anything,” https://github.com/luca-medeiros/lang-segment-anything , 2023

2023

Cited alongside, same era.

R. A. Potamias, J. Zhang, J. Deng, and S. Zafeiriou, “Wilor: End-to-end 3d hand localization and reconstruction in-the-wild,” 2024

2024

Later among the works it cites.

B. Wen, W. Yang, J. Kautz, and S. Birchfield, “Foundationpose: Unified 6d pose estimation and tracking of novel objects,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 17 868–17 879

2024

Later among the works it cites.

AR Code, “Ar code,” 2022, accessed: 2024-09-28. [Online]. Available: https://ar-code.com/

2024

Later among the works it cites.

G. Xu, R. Zheng, Y. Liang, X. Wang, Z. Yuan, T. Ji, Y. Luo, X. Liu, J. Yuan, P. Hua et al. , “Drm: Mastering visual reinforcement learning through dormant ratio minimization,” in The Twelfth International Conference on Learning Representations , 2024

2024

Later among the works it cites.