2019

Learning Latent Plans from Play

Lynch, Corey, Khansari, Mohi, Xiao, Ted et al.

Understand

Acquiring a diverse repertoire of general-purpose skills remains an open challenge for robotics.

In this work, we propose self-supervising control on top of human teleoperated play data as a way to scale up skill learning.
Play has two properties that make it attractive compared to conventional task demonstrations.
Play is cheap, as it can be collected in large quantities quickly without task segmenting, labeling, or resetting to an initial state.

Built on

Learning to achieve goals
L. P. Kaelbling · 1993
Earlier work this paper cites.
Learning and generalization of motor skills by learning from demonstration
P. Pastor, H. Hoffmann, T. Asfour, and S. Schaal · 2009
Earlier work this paper cites.
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Original
S. Ross, G. J. Gordon, and J. A. Bagnell · 2011
Earlier work this paper cites.
Reinforcement learning in robotics: A survey
J. Kober, J. A. Bagnell, and J. Peters · 2013
Earlier work this paper cites.
A survey on policy search for robotics
M. P. Deisenroth, G. Neumann, J. Peters, et al · 2013
Earlier work this paper cites.
Auto-encoding variational bayes
Original
D. P. Kingma and M. Welling · 2013
Earlier work this paper cites.
Learning structured output representation using deep conditional generative models
K. Sohn, H. Lee, and X. Yan · 2015
Earlier work this paper cites.
Mujoco haptix: A virtual reality system for hand manipulation
V. Kumar and E. Todorov · 2015
Earlier work this paper cites.
Learning to poke by poking: Experiential learning of intuitive physics
Original
P. Agrawal, A. Nair, P. Abbeel, J. Malik, and S. Levine · 2016
Earlier work this paper cites.
Transfer from simulation to real world through learning deep inverse dynamics model
Original
P. Christiano, Z. Shah, I. Mordatch, J. Schneider, T. Blackwell, J. Tobin, P. Abbeel, and W. Zaremba · 2016
Earlier work this paper cites.
Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours
L. Pinto and A. Gupta · 2016
Earlier work this paper cites.
Generating sentences from a continuous space
S. R. Bowman, L. Vilnis, O. Vinyals, A. M. Dai, R. Jozefowicz, and S. Bengio · 2016
Earlier work this paper cites.
Deep imitation learning for complex manipulation tasks from virtual reality teleoperation
Original
T. Zhang, Z. McCarthy, O. Jow, D. Lee, K. Goldberg, and P. Abbeel · 2017
Earlier work this paper cites.
Vision-based multi-task manipulation for inexpensive robots using end-to-end learning from demonstration
Original
R. Rahmatizadeh, P. Abolghasemi, L. Bölöni, and S. Levine · 2017
Earlier work this paper cites.

Similar

Learning complex dexterous manipulation with deep reinforcement learning and demonstrations
Original
A. Rajeswaran, V. Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S. Levine · 2017
Cited alongside, same era.
One-shot imitation learning
Original
Y. Duan, M. Andrychowicz, B. C. Stadie, J. Ho, J. Schneider, I. Sutskever, P. Abbeel, and W. Zaremba · 2017
Cited alongside, same era.
Hindsight experience replay
M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. Tobin, O. P. Abbeel, and W. Zaremba · 2017
Cited alongside, same era.
Hierarchical actor-critic
Original
A. Levy, R. P. Jr., and K. Saenko · 2017
Cited alongside, same era.
Hindsight policy gradients
Original
P. Rauber, F. Mutz, and J. Schmidhuber · 2017
Cited alongside, same era.
The intentional unintentional agent: Learning to solve many continuous control tasks simultaneously
Original
S. Cabi, S. G. Colmenarejo, M. W. Hoffman, M. Denil, Z. Wang, and N. de Freitas · 2017
Cited alongside, same era.
Combining self-supervised learning and imitation for vision-based rope manipulation
Original
A. Nair, D. Chen, P. Agrawal, P. Isola, P. Abbeel, J. Malik, and S. Levine · 2017
Cited alongside, same era.
Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection
S. Levine, P. Pastor, A. Krizhevsky, and D. Quillen · 2017
Cited alongside, same era.
One-shot visual imitation learning via meta-learning
Original
C. Finn, T. Yu, T. Zhang, P. Abbeel, and S. Levine · 2017
Cited alongside, same era.
Robust imitation of diverse behaviors
Z. Wang, J. S. Merel, S. E. Reed, N. de Freitas, G. Wayne, and N. Heess · 2017
Cited alongside, same era.
β \beta -VAE: Learning basic visual concepts with a constrained variational framework
I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner · 2017
Cited alongside, same era.
Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications
Original
T. Salimans, A. Karpathy, X. Chen, and D. P. Kingma · 2017
Cited alongside, same era.

Then

Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipulation
Original
D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke, et al · 2018
Later among the works it cites.
Soft actor-critic algorithms and applications
Original
T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel, et al · 2018
Later among the works it cites.
Visual foresight: Model-based deep reinforcement learning for vision-based robotic control
Original
F. Ebert, C. Finn, S. Dasari, A. Xie, A. Lee, and S. Levine · 2018
Later among the works it cites.
Unsupervised control through non-parametric discriminative rewards
Original
D. Warde-Farley, T. V. de Wiele, T. Kulkarni, C. Ionescu, S. Hansen, and V. Mnih · 2018
Later among the works it cites.
Reinforcement learning: An introduction
R. S. Sutton and A. G. Barto · 2018
Later among the works it cites.
Visual reinforcement learning with imagined goals
A. V. Nair, V. Pong, M. Dalal, S. Bahl, S. Lin, and S. Levine · 2018
Later among the works it cites.
Behavioral cloning from observation
Original
F. Torabi, G. Warnell, and P. Stone · 2018
Later among the works it cites.
Vision-based multi-task manipulation for inexpensive robots using end-to-end learning from demonstration
R. Rahmatizadeh, P. Abolghasemi, L. Boloni, and S. Levine · 2018
Later among the works it cites.
Zero-shot visual imitation
D. Pathak, P. Mahmoudieh, G. Luo, P. Agrawal, D. Chen, Y. Shentu, E. Shelhamer, J. Malik, A. A. Efros, and T. Darrell · 2018
Later among the works it cites.
Learning an embedding space for transferable robot skills
K. Hausman, J. T. Springenberg, Z. Wang, N. Heess, and M. Riedmiller · 2018
Later among the works it cites.
One-shot high-fidelity imitation: Training large-scale deep nets with RL
Original
T. L. Paine, S. G. Colmenarejo, Z. Wang, S. E. Reed, Y. Aytar, T. Pfaff, M. W. Hoffman, G. Barth-Maron, S. Cabi, D. Budden, and N. de Freitas · 2018
Later among the works it cites.
Near-optimal representation learning for hierarchical reinforcement learning
Original
O. Nachum, S. Gu, H. Lee, and S. Levine · 2018
Later among the works it cites.
Time-contrastive networks: Self-supervised learning from video
Original
P. Sermanet, C. Lynch, Y. Chebotar, J. Hsu, E. Jang, S. Schaal, and S. Levine · 2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…