2021

Towards More Generalizable One-shot Visual Imitation Learning

Mandi, Zhao, Liu, Fangchen, Lee, Kimin et al.

Understand

A general-purpose robot should be able to master a wide range of tasks and quickly learn a novel one by leveraging past experiences.

One-shot imitation learning (OSIL) approaches this goal by training an agent with (pairs of) expert demonstrations, such that at test time, it can directly execute a new task from just one demonstration.
However, so far this framework has been limited to training on many variations of one task, and testing on other unseen but similar variations of the same task.
In this work, we push for a higher level of generalization ability by investigating a more ambitious multi-task setup.

Built on

C. Lynch, M. Khansari, T. Xiao, V. Kumar, J. Tompson, S. Levine, and P. Sermanet, “Learning latent plans from play,” Conference on Robot Learning (CoRL) , 2019. [Online]. Available: https://arxiv.org/abs/1903.01973
Original
1903
Earlier work this paper cites.
T. Yu, D. Quillen, Z. He, R. Julian, K. Hausman, C. Finn, and S. Levine, “Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,” in Conference on Robot Learning (CoRL) , 2019. [Online]. Available: https://arxiv.org/abs/1910.10897
Original
1910
Earlier work this paper cites.
D. A. Pomerleau, “Efficient training of artificial neural networks for autonomous navigation,” Neural computation , vol. 3, no. 1, pp. 88–97, 1991
1991
Earlier work this paper cites.
P. K. Pook and D. H. Ballard, “Recognizing teleoperated manipulations,” in International Conference on Robotics and Automation , 1993
1993
Earlier work this paper cites.
M. Bain and C. Sammut, “A framework for behavioural cloning.” in Machine Intelligence 15 , 1995
1995
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation , vol. 9, no. 8, pp. 1735–1780, 1997
1997
Earlier work this paper cites.
D. Pomerleau, “An autonomous land vehicle in a neural network,” in Advances in Neural Information Processing Systems , 1998
1998
Earlier work this paper cites.
A. Y. Ng, S. J. Russell et al. , “Algorithms for inverse reinforcement learning.” in International Conference on Machine Learning , 2000
2000
Earlier work this paper cites.
P. Abbeel and A. Y. Ng, “Apprenticeship learning via inverse reinforcement learning,” in International Conference on Machine Learning , 2004
2004
Earlier work this paper cites.
C. Lynch and P. Sermanet, “Language conditioned imitation learning over unstructured data,” Robotics: Science and Systems , 2021. [Online]. Available: https://arxiv.org/abs/2005.07648
Original
2005
Earlier work this paper cites.
J. D. Sweeney and R. Grupen, “A model of shared grasp affordances from demonstration,” in International Conference on Humanoid Robots , 2007
2007
Earlier work this paper cites.
S. Ross, G. Gordon, and D. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning,” in International Conference on Artificial Intelligence and Statistics , 2011
2011
Earlier work this paper cites.
M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling, “The arcade learning environment: An evaluation platform for general agents,” Journal of Artificial Intelligence Research , vol. 47, pp. 253–279, 2013
2013
Earlier work this paper cites.
I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” arXiv preprint arXiv:1409.3215 , 2014
Original
2014
Earlier work this paper cites.
M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang et al. , “End to end learning for self-driving cars,” arXiv preprint arXiv:1604.07316 , 2016
Original
2016
Earlier work this paper cites.
J. Ho and S. Ermon, “Generative adversarial imitation learning,” arXiv preprint arXiv:1606.03476 , 2016
Original
2016
Earlier work this paper cites.

Similar

Y. Duan, M. Andrychowicz, B. C. Stadie, J. Ho, J. Schneider, I. Sutskever, P. Abbeel, and W. Zaremba, “One-shot imitation learning,” in Advances in Neural Information Processing Systems , 2017
2017
Cited alongside, same era.
C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International Conference on Machine Learning , 2017
2017
Cited alongside, same era.
C. Finn, T. Yu, T. Zhang, P. Abbeel, and S. Levine, “One-shot visual imitation learning via meta-learning,” in Conference on Robot Learning , 2017
2017
Cited alongside, same era.
T. Salimans, A. Karpathy, X. Chen, and D. P. Kingma, “Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications,” arXiv preprint arXiv:1701.05517 , 2017
Original
2017
Cited alongside, same era.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , 2017
2017
Cited alongside, same era.
Y. Aytar, T. Pfaff, D. Budden, T. L. Paine, Z. Wang, and N. de Freitas, “Playing hard exploration games by watching youtube,” in Advances in Neural Information Processing Systems , 2018
2018
Cited alongside, same era.
S. James, M. Bloesch, and A. J. Davison, “Task-embedded control networks for few-shot imitation learning,” in Conference on Robot Learning , 2018
2018
Cited alongside, same era.
——, “Task-embedded control networks for few-shot imitation learning,” 2018
2018
Cited alongside, same era.
X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 7794–7803
2018
Cited alongside, same era.
D. Xu, S. Nair, Y. Zhu, J. Gao, A. Garg, L. Fei-Fei, and S. Savarese, “Neural task programming: Learning to generalize across hierarchical tasks,” 2018
2018
Cited alongside, same era.
T. Yu, C. Finn, A. Xie, S. Dasari, T. Zhang, P. Abbeel, and S. Levine, “One-shot imitation from observing humans via domain-adaptive meta-learning,” in Robotics: Science and Systems , 2018
2018
Cited alongside, same era.
D.-A. Huang, S. Nair, D. Xu, Y. Zhu, A. Garg, L. Fei-Fei, S. Savarese, and J. C. Niebles, “Neural task graphs: Generalizing to unseen tasks from a single video demonstration,” 2019
2019
Cited alongside, same era.

Then

S. Dasari and A. Gupta, “Transformers for one-shot visual imitation,” in Conference on Robot Learning , 2020
2020
Later among the works it cites.
J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Doersch, B. A. Pires, Z. D. Guo, M. G. Azar, B. Piot, K. Kavukcuoglu, R. Munos, and M. Valko, “Bootstrap your own latent: A new approach to self-supervised learning,” 2020
2020
Later among the works it cites.
K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020
2020
Later among the works it cites.
O. Henaff, “Data-efficient image recognition with contrastive predictive coding,” in International Conference on Machine Learning , 2020
2020
Later among the works it cites.
O. J. Hénaff, A. Srinivas, J. Fauw, A. Razavi, C. Doersch, S. Eslami, and A. van den Oord, “Data-efficient image recognition with contrastive predictive coding,” ArXiv , vol. abs/1905.09272, 2020
Original
2020
Later among the works it cites.
M. Laskin, A. Srinivas, and P. Abbeel, “Curl: Contrastive unsupervised representations for reinforcement learning,” in International Conference on Machine Learning , 2020
2020
Later among the works it cites.
——, “Learning latent plans from play,” in Conference on Robot Learning . PMLR, 2020, pp. 1113–1132
2020
Later among the works it cites.
X. B. Peng, E. Coumans, T. Zhang, T.-W. Lee, J. Tan, and S. Levine, “Learning agile robotic locomotion skills by imitating animals,” in Robotics: Science and Systems , 2020
2020
Later among the works it cites.
A. Singh, E. Jang, A. Irpan, D. Kappler, M. Dalal, S. Levine, M. Khansari, and C. Finn, “Scalable multi-task imitation learning with autonomous improvement,” 2020
2020
Later among the works it cites.
S. Stepputtis, J. Campbell, M. Phielipp, S. Lee, C. Baral, and H. Ben Amor, “Language-conditioned imitation learning for robot manipulation tasks,” in Advances in Neural Information Processing Systems , H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 13 139–13 150. [Online]. Available: https://proceedings.neurips.cc/paper/2020/file/9909794d52985cbc5d95c26e31125d1a-Paper.pdf
2020
Later among the works it cites.
S. Young, D. Gandhi, S. Tulsiani, A. Gupta, P. Abbeel, and L. Pinto, “Visual imitation made easy,” in Conference on Robot Learning , 2020
2020
Later among the works it cites.
A. Zhou, E. Jang, D. Kappler, A. Herzog, M. Khansari, P. Wohlhart, Y. Bai, M. Kalakrishnan, S. Levine, and C. Finn, “Watch, try, learn: Meta-learning from demonstrations and reward,” 2020
2020
Later among the works it cites.
Y. Zhu, J. Wong, A. Mandlekar, and R. Martín-Martín, “robosuite: A modular simulation framework and benchmark for robot learning,” arXiv preprint arXiv:2009.12293 , 2020
Original
2020
Later among the works it cites.
——, “Bc-0: Zero-shot task generalization with robotic imitation learning,” in Conference on Robot Learning , 2021
2021
Closest in time.
M. Schwarzer, A. Anand, R. Goel, R. D. Hjelm, A. Courville, and P. Bachman, “Data-efficient reinforcement learning with self-predictive representations,” in International Conference on Learning Representations , 2021
2021
Closest in time.
A. Stooke, K. Lee, P. Abbeel, and M. Laskin, “Decoupling representation learning from reinforcement learning,” in International Conference on Machine Learning , 2021
2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…