Built on

D. Rakita, B. Mutlu, and M. Gleicher, “Relaxedik: Real-time synthesis of accurate and feasible robot arm motion.” in Robotics: Science and Systems , vol. 14. Pittsburgh, PA, 2018, pp. 26–30
2018
Earlier work this paper cites.
I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations , 2019. [Online]. Available: https://openreview.net/forum?id=Bkg6RiCqY7
2019
Earlier work this paper cites.
J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” arXiv preprint arXiv:2010.02502 , 2020
Original
2020
Earlier work this paper cites.
A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu et al. , “Rt-1: Robotics transformer for real-world control at scale,” arXiv preprint arXiv:2212.06817 , 2022
Original
2022
Earlier work this paper cites.
S. Nair, A. Rajeswaran, V. Kumar, C. Finn, and A. Gupta, “R3m: A universal visual representation for robot manipulation,” arXiv preprint arXiv:2203.12601 , 2022
Original
2022
Earlier work this paper cites.
N. Hansen, Z. Yuan, Y. Ze, T. Mu, A. Rajeswaran, H. Su, H. Xu, and X. Wang, “On pre-training for visuo-motor control: Revisiting a learning-from-scratch baseline,” arXiv preprint arXiv:2212.05749 , 2022
Original
2022
Earlier work this paper cites.
Y. Qin, W. Yang, B. Huang, K. Van Wyk, H. Su, X. Wang, Y.-W. Chao, and D. Fox, “Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system,” in Robotics: Science and Systems , 2023
2023
Earlier work this paper cites.
C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” arXiv preprint arXiv:2303.04137 , 2023
Original
2023
Earlier work this paper cites.
T. Z. Zhao, V. Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” arXiv preprint arXiv:2304.13705 , 2023
Original
2023
Earlier work this paper cites.
A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn et al. , “Rt-2: Vision-language-action models transfer web knowledge to robotic control,” arXiv preprint arXiv:2307.15818 , 2023
Original
2023
Earlier work this paper cites.
M. Seo, S. Han, K. Sim, S. H. Bang, C. Gonzalez, L. Sentis, and Y. Zhu, “Deep imitation learning for humanoid loco-manipulation through human teleoperation,” in IEEE-RAS International Conference on Humanoid Robots (Humanoids) , 2023
2023
Earlier work this paper cites.
Y. Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu, “3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations,” arXiv preprint arXiv:2403.03954 , 2024
Original
2024
Earlier work this paper cites.
R. Ding, Y. Qin, J. Zhu, C. Jia, S. Yang, R. Yang, X. Qi, and X. Wang, “Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning,” arXiv preprint arXiv:2407.03162 , 2024
Original
2024
Earlier work this paper cites.
S. Yang, M. Liu, Y. Qin, R. Ding, J. Li, X. Cheng, R. Yang, S. Yi, and X. Wang, “Ace: A cross-platform visual-exoskeletons system for low-cost dexterous teleoperation,” arXiv preprint arXiv:2408.11805 , 2024
Original
2024
Earlier work this paper cites.
K. Shaw, Y. Li, J. Yang, M. K. Srirama, R. Liu, H. Xiong, R. Mendonca, and D. Pathak, “Bimanual dexterity for complex tasks,” in 8th Annual Conference on Robot Learning , 2024
2024
Earlier work this paper cites.

Similar

T. He, Z. Luo, X. He, W. Xiao, C. Zhang, W. Zhang, K. Kitani, C. Liu, and G. Shi, “Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning,” in arXiv , 2024
2024
Cited alongside, same era.
Z. Fu, Q. Zhao, Q. Wu, G. Wetzstein, and C. Finn, “Humanplus: Humanoid shadowing and imitation from humans,” in arXiv , 2024
2024
Cited alongside, same era.
T. Lin, Z.-H. Yin, H. Qi, P. Abbeel, and J. Malik, “Twisting lids off with two hands,” arXiv:2403.02338 , 2024
Original
2024
Cited alongside, same era.
Z. Yuan, T. Wei, S. Cheng, G. Zhang, Y. Chen, and H. Xu, “Learning to manipulate anywhere: A visual generalizable framework for reinforcement learning,” arXiv preprint arXiv:2407.15815 , 2024
Original
2024
Cited alongside, same era.
X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang, “Open-television: Teleoperation with immersive active visual feedback,” arXiv preprint arXiv:2407.01512 , 2024

Then

I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath, “Real-world humanoid locomotion with reinforcement learning,” Science Robotics , vol. 9, no. 89, p. eadi9579, 2024
2024
Closest in time.
Z. Zhuang, S. Yao, and H. Zhao, “Humanoid parkour learning,” arXiv preprint arXiv:2406.10759 , 2024
Original
2024
Closest in time.
X. Cheng, Y. Ji, J. Chen, R. Yang, G. Yang, and X. Wang, “Expressive whole-body control for humanoid robots,” arXiv preprint arXiv:2402.16796 , 2024
Original
2024
Closest in time.
C. Wang, H. Shi, W. Wang, R. Zhang, L. Fei-Fei, and C. K. Liu, “Dexcap: Scalable and portable mocap data collection system for dexterous manipulation,” arXiv preprint arXiv:2403.07788 , 2024
Original
2024
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Generalizable Humanoid Manipulation with 3D Diffusion Policies

Ze, Yanjie, Chen, Zixuan, Wang, Wenhao et al.

Understand

Humanoid robots capable of autonomous operation in diverse environments have long been a goal for roboticists.

However, autonomous manipulation by humanoid robots has largely been restricted to one specific scene, primarily due to the difficulty of acquiring generalizable skills and the expensiveness of in-the-wild humanoid robot data.
In this work, we build a real-world robotic system to address this challenging problem.
Our system is mainly an integration of 1) a whole-upper-body robotic teleoperation system to acquire human-like robot data, 2) a 25-DoF humanoid robot platform with a height-adjustable cart and a 3D LiDAR sensor, and 3) an improved 3D Diffusion Policy learning algorithm for humanoid robots to learn from noisy human data.

Built on

D. Rakita, B. Mutlu, and M. Gleicher, “Relaxedik: Real-time synthesis of accurate and feasible robot arm motion.” in Robotics: Science and Systems , vol. 14. Pittsburgh, PA, 2018, pp. 26–30
2018
Earlier work this paper cites.
I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations , 2019. [Online]. Available: https://openreview.net/forum?id=Bkg6RiCqY7
2019
Earlier work this paper cites.
J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” arXiv preprint arXiv:2010.02502 , 2020
Original
2020
Earlier work this paper cites.
A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu et al. , “Rt-1: Robotics transformer for real-world control at scale,” arXiv preprint arXiv:2212.06817 , 2022
Original
2022
Earlier work this paper cites.
S. Nair, A. Rajeswaran, V. Kumar, C. Finn, and A. Gupta, “R3m: A universal visual representation for robot manipulation,” arXiv preprint arXiv:2203.12601 , 2022
Original
2022
Earlier work this paper cites.
N. Hansen, Z. Yuan, Y. Ze, T. Mu, A. Rajeswaran, H. Su, H. Xu, and X. Wang, “On pre-training for visuo-motor control: Revisiting a learning-from-scratch baseline,” arXiv preprint arXiv:2212.05749 , 2022
Original
2022
Earlier work this paper cites.
Y. Qin, W. Yang, B. Huang, K. Van Wyk, H. Su, X. Wang, Y.-W. Chao, and D. Fox, “Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system,” in Robotics: Science and Systems , 2023
2023
Earlier work this paper cites.
C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” arXiv preprint arXiv:2303.04137 , 2023
Original
2023
Earlier work this paper cites.
T. Z. Zhao, V. Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” arXiv preprint arXiv:2304.13705 , 2023
Original
2023
Earlier work this paper cites.
A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn et al. , “Rt-2: Vision-language-action models transfer web knowledge to robotic control,” arXiv preprint arXiv:2307.15818 , 2023
Original
2023
Earlier work this paper cites.
M. Seo, S. Han, K. Sim, S. H. Bang, C. Gonzalez, L. Sentis, and Y. Zhu, “Deep imitation learning for humanoid loco-manipulation through human teleoperation,” in IEEE-RAS International Conference on Humanoid Robots (Humanoids) , 2023
2023
Earlier work this paper cites.
Y. Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu, “3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations,” arXiv preprint arXiv:2403.03954 , 2024
Original
2024
Earlier work this paper cites.
R. Ding, Y. Qin, J. Zhu, C. Jia, S. Yang, R. Yang, X. Qi, and X. Wang, “Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning,” arXiv preprint arXiv:2407.03162 , 2024
Original
2024
Earlier work this paper cites.
S. Yang, M. Liu, Y. Qin, R. Ding, J. Li, X. Cheng, R. Yang, S. Yi, and X. Wang, “Ace: A cross-platform visual-exoskeletons system for low-cost dexterous teleoperation,” arXiv preprint arXiv:2408.11805 , 2024
Original
2024
Earlier work this paper cites.
K. Shaw, Y. Li, J. Yang, M. K. Srirama, R. Liu, H. Xiong, R. Mendonca, and D. Pathak, “Bimanual dexterity for complex tasks,” in 8th Annual Conference on Robot Learning , 2024
2024
Earlier work this paper cites.

Similar

T. He, Z. Luo, X. He, W. Xiao, C. Zhang, W. Zhang, K. Kitani, C. Liu, and G. Shi, “Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning,” in arXiv , 2024
2024
Cited alongside, same era.
Z. Fu, Q. Zhao, Q. Wu, G. Wetzstein, and C. Finn, “Humanplus: Humanoid shadowing and imitation from humans,” in arXiv , 2024
2024
Cited alongside, same era.
T. Lin, Z.-H. Yin, H. Qi, P. Abbeel, and J. Malik, “Twisting lids off with two hands,” arXiv:2403.02338 , 2024
Original
2024
Cited alongside, same era.
Z. Yuan, T. Wei, S. Cheng, G. Zhang, Y. Chen, and H. Xu, “Learning to manipulate anywhere: A visual generalizable framework for reinforcement learning,” arXiv preprint arXiv:2407.15815 , 2024
Original
2024
Cited alongside, same era.
X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang, “Open-television: Teleoperation with immersive active visual feedback,” arXiv preprint arXiv:2407.01512 , 2024

Then

I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath, “Real-world humanoid locomotion with reinforcement learning,” Science Robotics , vol. 9, no. 89, p. eadi9579, 2024
2024
Closest in time.
Z. Zhuang, S. Yao, and H. Zhao, “Humanoid parkour learning,” arXiv preprint arXiv:2406.10759 , 2024
Original
2024
Closest in time.
X. Cheng, Y. Ji, J. Chen, R. Yang, G. Yang, and X. Wang, “Expressive whole-body control for humanoid robots,” arXiv preprint arXiv:2402.16796 , 2024
Original
2024
Closest in time.
C. Wang, H. Shi, W. Wang, R. Zhang, L. Fei-Fei, and C. K. Liu, “Dexcap: Scalable and portable mocap data collection system for dexterous manipulation,” arXiv preprint arXiv:2403.07788 , 2024
Original
2024
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

2024

Cited alongside, same era.

Boston Dynamics, “Atlas,” 2024, online. [Online]. Available: https://bostondynamics.com/atlas/

2024

Cited alongside, same era.

Tesla, “Optimus,” 2024, online. [Online]. Available: https://www.tesla.com/en_eu/AI

2024

Cited alongside, same era.

Figure, “01,” 2024, online. [Online]. Available: https://www.figure.ai/

2024

Cited alongside, same era.

Unitree, “H1,” 2024, online. [Online]. Available: https://www.unitree.com/h1

2024

Cited alongside, same era.

Fourier Intelligence, “Gr1,” 2024, online. [Online]. Available: https://www.fourierintelligence.com/gr1

2024

Cited alongside, same era.

T. He, Z. Luo, W. Xiao, C. Zhang, K. Kitani, C. Liu, and G. Shi, “Learning human-to-humanoid real-time whole-body teleoperation,” in arXiv , 2024

2024

Cited alongside, same era.

Z. Fu, T. Z. Zhao, and C. Finn, “Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,” in arXiv , 2024

2024

Cited alongside, same era.

H. Etukuru, N. Naka, Z. Hu, J. Mehu, A. Edsinger, C. Paxton, S. Chintala, L. Pinto, and N. M. M. Shafiullah, “General policies for zero-shot deployment in new environments,” arXiv , 2024

2024

Closest in time.

Inspire Robots, “Dexterous hands,” 2024, online. [Online]. Available: http://www.inspire-robots.store/collections/the-dexterous-hands

2024

Closest in time.

Intel RealSense, “Lidar camera l515,” 2024, online. [Online]. Available: https://www.intelrealsense.com/lidar-camera-l515/

2024

Closest in time.

——, “Depth camera d435,” 2024, online. [Online]. Available: https://www.intelrealsense.com/depth-camera-d435/

2024

Closest in time.

C. Wang, H. Fang, H.-S. Fang, and C. Lu, “Rise: 3d perception makes real-world robot imitation simple and effective,” arXiv preprint arXiv:2404.12281 , 2024

Original

2024

Closest in time.

Apple, “Apple vision pro,” 2024, online. [Online]. Available: https://www.apple.com/apple-vision-pro/

2024

Closest in time.

Y. Park and P. Agrawal, “Using apple vision pro to train and control robots,” 2024, online. [Online]. Available: https://github.com/Improbable-AI/VisionProTeleop

2024

Closest in time.

J. Yang, Z. ang Cao, C. Deng, R. Antonova, S. Song, and J. Bohg, “Equibot: Sim(3)-equivariant diffusion policy for generalizable and data efficient learning,” arXiv preprint arXiv:2407.01479 , 2024

Original

2024

Closest in time.

S. Tian, B. Wulfe, K. Sargent, K. Liu, S. Zakharov, V. Guizilini, and J. Wu, “View-invariant policy learning via zero-shot novel view synthesis,” arXiv preprint arXiv:2409.03685 , 2024

Original

2024

Closest in time.

D. Wang, S. Hart, D. Surovik, T. Kelestemur, H. Huang, H. Zhao, M. Yeatman, J. Wang, R. Walters, and R. Platt, “Equivariant diffusion policy,” arXiv preprint arXiv:2407.01812 , 2024

Original

2024

Closest in time.