2022

Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation

Arunachalam, Sridhar Pandian, Silwal, Sneha, Evans, Ben et al.

Understand

Optimizing behaviors for dexterous manipulation has been a longstanding challenge in robotics, with a variety of methods from model-based control to model-free reinforcement learning having been previously explored in literature.

Perhaps one of the most powerful techniques to learn complex manipulation strategies is imitation learning.
However, collecting and learning from demonstrations in dexterous manipulation is quite challenging.
The complex, high-dimensional action-space involved with multi-finger control often leads to poor sample efficiency of learning-based methods.

Built on

H. Moravec,
1988
Earlier work this paper cites.
D. A. Pomerleau, “Alvinn: An autonomous land vehicle in a neural network,” in
1988
Earlier work this paper cites.
E. Fix and J. L. Hodges,
1989
Earlier work this paper cites.
D. A. Pomerleau, “Alvinn: An autonomous land vehicle in a neural network,” in
1989
Earlier work this paper cites.
S. Schaal and C. Atkeson, “Robot juggling: implementation of memory-based learning,”
1994
Earlier work this paper cites.
P. Bakker and Y. Kuniyoshi, “Robot see, robot do: An overview of robot imitation,” in
1996
Earlier work this paper cites.
S. Schaal, “Is imitation learning the route to humanoid robots?”
1999
Earlier work this paper cites.
S. M. Kakade, “A natural policy gradient,” in
2001
Earlier work this paper cites.
L. Biagiotti, F. Lotti, C. Melchiorri, and G. Vassura, “How far is the human hand,” 2004
2004
Earlier work this paper cites.
P. Abbeel and A. Y. Ng, “Apprenticeship learning via inverse reinforcement learning,” in
2004
Earlier work this paper cites.
A. M. Dollar and R. D. Howe, “The sdm hand as a prosthetic terminal device: a feasibility study,” in
2007
Earlier work this paper cites.
U. Syed and R. E. Schapire, “A game-theoretic approach to apprenticeship learning,” in
2007
Earlier work this paper cites.
R. Y. Wang and J. Popović, “Real-time hand-tracking with a color glove,”
2009
Earlier work this paper cites.
M. R. Dogar and S. S. Srinivasa, “Push-grasping with dexterous hands: Mechanics and a method,” in
2010
Earlier work this paper cites.
S. Ross, G. J. Gordon, and J. A. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning,” 2011
2011
Earlier work this paper cites.
B. Akgun, M. Cakmak, K. Jiang, and A. L. Thomaz, “Keyframe-based learning from demonstration,”
2012
Earlier work this paper cites.
E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A physics engine for model-based control,” in
2012
Earlier work this paper cites.
S. Andrews and P. G. Kry, “Goal directed multi-finger manipulation: Control policies and analysis,”
2013
Earlier work this paper cites.
Y. Bai and C. K. Liu, “Dexterous manipulation using both palm and fingers,” in
2014
Earlier work this paper cites.

Similar

K. Muelling, A. Boularias, B. Mohler, B. Schölkopf, and J. Peters, “Learning strategies in table tennis using inverse reinforcement learning,”
2014
Cited alongside, same era.
T. Schmidt, R. A. Newcombe, and D. Fox, “Dart: Dense articulated real-time tracking,” in
2014
Cited alongside, same era.
I. Lenz, R. A. Knepper, and A. Saxena, “Deepmpc: Learning deep latent features for model predictive control.” in
2015
Cited alongside, same era.
V. Kumar, A. Gupta, E. Todorov, and S. Levine, “Learning dexterous manipulation policies from experience and imitation,”
2016
Cited alongside, same era.
K. Tai, A.-R. El-Sayed, M. Shahriari, M. Biglarbegian, and S. Mahmud, “State of the art robotic grippers and applications,”
2016
Cited alongside, same era.
M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang,
Original
2016
Cited alongside, same era.
V. Kumar, E. Todorov, and S. Levine, “Optimal control with learned local models: Application to dexterous manipulation,” in
2016
Cited alongside, same era.
G. Moon, J. Y. Chang, and K. M. Lee, “V2v-posenet: Voxel-to-voxel prediction network for accurate 3d hand and human pose estimation from a single depth map,”
2017
Cited alongside, same era.
M. Vecerík, T. Hester, J. Scholz, F. Wang, O. Pietquin, B. Piot, N. Heess, T. Rothörl, T. Lampe, and M. A. Riedmiller, “Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards,”
2017
Cited alongside, same era.
A. Nair, D. Chen, P. Agrawal, P. Isola, P. Abbeel, J. Malik, and S. Levine, “Combining self-supervised learning and imitation for vision-based rope manipulation,” in
2017
Cited alongside, same era.
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” 2017
2017
Cited alongside, same era.
OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Józefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba, “Learning dexterous in-hand manipulation,”
2018
Cited alongside, same era.

Then

OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba, “Learning dexterous in-hand manipulation,” 2019
2019
Later among the works it cites.
C. Lynch, M. Khansari, T. Xiao, V. Kumar, J. Tompson, S. Levine, and P. Sermanet, “Learning latent plans from play,” 2019
2019
Later among the works it cites.
F. Zhang, V. Bazarevsky, A. Vakunov, A. Tkachenka, G. Sung, C.-L. Chang, and M. Grundmann, “Mediapipe hands: On-device real-time hand tracking,” 2020
2020
Later among the works it cites.
S. Song, A. Zeng, J. Lee, and T. Funkhouser, “Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations,”
2020
Later among the works it cites.
S. Young, D. Gandhi, S. Tulsiani, A. Gupta, P. Abbeel, and L. Pinto, “Visual imitation made easy,” 2020
2020
Later among the works it cites.
L. Guan Ming, J. Prayook, and A. Wei Tech, “Mobilehand: Real-time 3d hand shape and pose estimation from color image,” in
2020
Later among the works it cites.
A. Zhan, P. Zhao, L. Pinto, P. Abbeel, and M. Laskin, “A framework for efficient robotic manipulation,”
Original
2020
Later among the works it cites.
J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar,
2020
Later among the works it cites.
M. Ahn, H. Zhu, K. Hartikainen, H. Ponte, A. Gupta, S. Levine, and V. Kumar, “Robel: Robotics benchmarks for learning with low-cost robots,” in
2020
Later among the works it cites.
T. Chen, J. Xu, and P. Agrawal, “A system for general in-hand object re-orientation,”
2021
Later among the works it cites.
M. Caeiro-Rodríguez, I. Otero-González, F. A. Mikic-Fonte, and M. Llamas-Nistal, “A systematic review of commercial smart gloves: Current status and applications,”
2021
Later among the works it cites.
J. Pari, N. M. Shafiullah, S. P. Arunachalam, and L. Pinto, “The surprising effectiveness of representation learning for visual imitation,” 2021
2021
Later among the works it cites.
Y. Qin, Y. Wu, S. Liu, H. Jiang, R. Yang, Y. Fu, and X. Wang, “Dexmv: Imitation learning for dexterous manipulation from human videos,”
2021
Later among the works it cites.
P. Mandikal and K. Grauman, “Dexvip: Learning dexterous grasping with human hand pose priors from video,” in
2021
Later among the works it cites.
P. Florence, C. Lynch, A. Zeng, O. Ramirez, A. Wahid, L. Downs, A. Wong, J. Lee, I. Mordatch, and J. Tompson, “Implicit behavioral cloning,” 2021
2021
Later among the works it cites.
S. Young, J. Pari, P. Abbeel, and L. Pinto, “Playful interactions for representation learning,”
2021
Later among the works it cites.
I. Radosavovic, X. Wang, L. Pinto, and J. Malik, “State-only imitation learning for dexterous manipulation,” in
2021
Later among the works it cites.
A. Sivakumar, K. Shaw, and D. Pathak, “Robotic telekinesis: Learning a robotic hand imitator by watching humans on youtube,” 2022
2022
Closest in time.
D. Yarats, D. Brandfonbrener, H. Liu, M. Laskin, P. Abbeel, A. Lazaric, and L. Pinto, “Don’t change the algorithm, change the data: Exploratory data for offline reinforcement learning,” 2022
2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…