2018

Zero-Shot Visual Imitation

Pathak, Deepak, Mahmoudieh, Parsa, Luo, Guanghao et al.

Understand

The current dominant paradigm for imitation learning relies on strong supervision of expert actions to learn both 'what' and 'how' to imitate.

We pursue an alternative paradigm wherein an agent first explores the world without any expert supervision and then distills its experience into a goal-conditioned skill policy with a novel forward consistency loss.
In our framework, the role of the expert is only to communicate the goals (i.e., what to imitate) during inference.
The learned policy is then employed to mimic the expert (i.e., how to imitate) after seeing just a sequence of images demonstrating the desired task.

Built on

Social learning theory , volume 1
Albert Bandura and Richard H Walters · 1977
Earlier work this paper cites.
Teaching by showing: Generating robot programs by visual observation of human performance
Yasuo Kuniyoshi, Masayuki Inaba, and Hirockika Inoue · 1989
Earlier work this paper cites.
ALVINN: An autonomous land vehicle in a neural network
Dean A Pomerleau · 1989
Earlier work this paper cites.
A possibility for implementing curiosity and boredom in model-building neural controllers
Jurgen Schmidhuber · 1991
Earlier work this paper cites.
Forward models: Supervised learning with a distal teacher
Michael I Jordan and David E Rumelhart · 1992
Earlier work this paper cites.
Visual servoing: real-time control of robot manipulators based on visual sensory feedback , volume 7
Hashimoto Koichi and Husband Tom · 1993
Earlier work this paper cites.
Toward an assembly plan from observation. i. task recognition with polyhedral objects
Katsushi Ikeuchi and Takashi Suehiro · 1994
Earlier work this paper cites.
Learning by watching: Extracting reusable task knowledge from visual observation of human performance
Yasuo Kuniyoshi, Masayuki Inaba, and Hirochika Inoue · 1994
Earlier work this paper cites.
Active, uncalibrated visual servoing
Billibon H Yoshimi and Peter K Allen · 1994
Earlier work this paper cites.
An internal model for sensorimotor integration
Daniel M Wolpert, Zoubin Ghahramani, and Michael I Jordan · 1995
Earlier work this paper cites.
Relative end-effector control using cartesian position based visual servoing
William J Wilson, CC Williams Hulls, and Graham S Bell · 1996
Earlier work this paper cites.
Mobile robot localisation using active vision
Andrew J Davison and David W Murray · 1998
Earlier work this paper cites.
Is imitation learning the route to humanoid robots?
Stefan Schaal · 1999
Earlier work this paper cites.
Algorithms for inverse reinforcement learning
Andrew Y Ng and Stuart J Russell · 2000
Earlier work this paper cites.
Robots that imitate humans
Cynthia Breazeal and Brian Scassellati · 2002
Earlier work this paper cites.
Structure in the space of value functions
David Foster and Peter Dayan · 2002
Earlier work this paper cites.
A new point matching algorithm for non-rigid registration
Haili Chui and Anand Rangarajan · 2003
Earlier work this paper cites.
Apprenticeship learning via inverse reinforcement learning
Pieter Abbeel and Andrew Y. Ng · 2004
Earlier work this paper cites.

Similar

Teaching and learning of robot tasks via observation of human performance
Rüdiger Dillmann · 2004
Cited alongside, same era.
Intrinsic motivation systems for autonomous mental development
Pierre-Yves Oudeyer, Frdric Kaplan, and Verena V Hafner · 2007
Cited alongside, same era.
Maximum entropy inverse reinforcement learning
Brian D. Ziebart, Andrew Maas, J Andrew Bagnell, and Anind K Dey · 2008
Cited alongside, same era.
A survey of robot learning from demonstration
Brenna D Argall, Sonia Chernova, Manuela Veloso, and Brett Browning · 2009
Cited alongside, same era.
Photometric visual servoing for omnidirectional cameras
Guillaume Caron, Eric Marchand, and El Mustapha Mouaddib · 2013
Cited alongside, same era.
Auto-encoding variational bayes
Original
Diederik P Kingma and Max Welling · 2013
Cited alongside, same era.
Acquiring visual servoing reaching and grasping skills using neural reinforcement learning
Thomas Lampe and Martin Riedmiller · 2013
Cited alongside, same era.
Stochastic backpropagation and approximate inference in deep generative models
Original
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra · 2014
Cited alongside, same era.
Adam: A method for stochastic optimization
Diederik Kingma and Jimmy Ba · 2015
Cited alongside, same era.
Action-conditional video prediction using deep networks in atari games
Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, Richard L Lewis, and Satinder Singh · 2015
Cited alongside, same era.
Universal value function approximators
Tom Schaul, Daniel Horgan, Karol Gregor, and David Silver · 2015
Cited alongside, same era.
Embed to control: A locally linear latent dynamics model for control from raw images
Manuel Watter, Jost Springenberg, Joschka Boedecker, and Martin Riedmiller · 2015
Cited alongside, same era.

Then

Open source structure from motion pipeline
Mapillary · 2016
Later among the works it cites.
Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours
Lerrel Pinto and Abhinav Gupta · 2016
Later among the works it cites.
Learning dense correspondence via 3d-guided cycle consistency
Tinghui Zhou, Philipp Krahenbuhl, Mathieu Aubry, Qixing Huang, and Alexei A. Efros · 2016
Later among the works it cites.
Hindsight experience replay
Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, Pieter Abbeel, and Wojciech Zaremba · 2017
Later among the works it cites.
One-shot imitation learning
Yan Duan, Marcin Andrychowicz, Bradly Stadie, OpenAI Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, and Wojciech Zaremba · 2017
Later among the works it cites.
Self-supervised visual planning with temporal skip connections
Original
Frederik Ebert, Chelsea Finn, Alex X Lee, and Sergey Levine · 2017
Later among the works it cites.
One-shot visual imitation learning via meta-learning
Chelsea Finn, Tianhe Yu, Tianhao Zhang, Pieter Abbeel, and Sergey Levine · 2017
Later among the works it cites.
Cognitive mapping and planning for visual navigation
Saurabh Gupta, James Davidson, Sergey Levine, Rahul Sukthankar, and Jitendra Malik · 2017
Later among the works it cites.
Learning visual servoing with deep features and fitted q-iteration
Original
Alex X Lee, Sergey Levine, and Pieter Abbeel · 2017
Later among the works it cites.
Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras
Raul Mur-Artal and Juan D Tardós · 2017
Later among the works it cites.
Combining self-supervised learning and imitation for vision-based rope manipulation
Ashvin Nair, Dian Chen, Pulkit Agrawal, Phillip Isola, Pieter Abbeel, Jitendra Malik, and Sergey Levine · 2017
Later among the works it cites.
Curiosity-driven exploration by self-supervised prediction
Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell · 2017
Later among the works it cites.
Unsupervised perceptual rewards for imitation learning
Pierre Sermanet, Kelvin Xu, and Sergey Levine · 2017
Later among the works it cites.
Third-person imitation learning
Bradly C. Stadie, Pieter Abbeel, and Ilya Sutskever · 2017
Later among the works it cites.
Unpaired image-to-image translation using cycle-consistent adversarial networks
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros · 2017
Later among the works it cites.
Imitation from observation: Learning to imitate behaviors from raw video via context translation
YuXuan Liu, Abhishek Gupta, Pieter Abbeel, and Sergey Levine · 2018
Closest in time.
Time-contrastive networks: Self-supervised learning from video
Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, and Sergey Levine · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…