2017

Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstration

Rahmatizadeh, Rouhollah, Abolghasemi, Pooya, Bölöni, Ladislau et al.

Understand

We propose a technique for multi-task learning from demonstration that trains the controller of a low-cost robotic arm to accomplish several complex picking and placing tasks, as well as non-prehensile manipulation.

The controller is a recurrent neural network using raw images as input and generating robot arm trajectories, with the parameters shared across the tasks.
The controller also combines VAE-GAN-based reconstruction with autoregressive multimodal action prediction.
Our results demonstrate that it is possible to learn complex manipulation tasks, such as picking up a towel, wiping an object, and depositing the towel to its previous position, entirely from raw images with direct behavior cloning.

Built on

G. J. McLachlan and K. E. Basford, “Mixture models. inference and applications to clustering,” Statistics: Textbooks and Monographs, New York: Dekker
1988
Earlier work this paper cites.
R. M. Neal, “Connectionist learning of belief networks,” Artificial intelligence
1992
Earlier work this paper cites.
C. M. Bishop, “Mixture density networks,” Technical Report
1994
Earlier work this paper cites.
R. Caruana, “Learning many related tasks at the same time with backpropagation.,” Advances in neural information processing systems
1995
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation
1997
Earlier work this paper cites.
Y. Bengio and S. Bengio, “Modeling high-dimensional discrete data with multi-layer neural networks.,” in Advances in neural information processing systems (NIPS)
1999
Earlier work this paper cites.
H. Mayer, F. Gomez, D. Wierstra, I. Nagy, A. Knoll, and J. Schmidhuber, “A system for robotic heart surgery that learns to tie knots using recurrent neural networks,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
2006
Earlier work this paper cites.
S. Calinon, F. Guenter, and A. Billard, “On learning, representing, and generalizing a task in a humanoid robot,” IEEE Transactions on Systems, Man, and Cybernetics
2007
Earlier work this paper cites.
J. Peters and S. Schaal, “Reinforcement learning of motor skills with policy gradients,” Neural networks
2008
Earlier work this paper cites.
P. Pastor, H. Hoffmann, T. Asfour, and S. Schaal, “Learning and generalization of motor skills by learning from demonstration,” in IEEE International Conference on Robotics and Automation (ICRA)
2009
Earlier work this paper cites.
B. D. Argall, S. Chernova, M. Veloso, and B. Browning, “A survey of robot learning from demonstration,” Robotics and autonomous systems
2009
Earlier work this paper cites.
S. Calinon, F. D’halluin, D. G. Caldwell, and A. Billard, “Handling of multiple constraints and motion alternatives in a robot programming by demonstration framework.,” in IEEE International Conference on Humanoid Robots (Humanoids)
2009
Earlier work this paper cites.
P. Abbeel, A. Coates, and A. Y. Ng, “Autonomous helicopter aerobatics through apprenticeship learning,” International Journal of Robotics Research (IJRR)
2010
Earlier work this paper cites.
S. Calinon, F. D’halluin, E. L. Sauser, D. G. Caldwell, and A. G. Billard, “Learning and reproduction of gestures by imitation,” IEEE Robotics & Automation Magazine
2010
Earlier work this paper cites.
C. Crick, S. Osentoski, G. Jay, and O. C. Jenkins, “Human and robot perception in large-scale learning from demonstration,” in International conference on Human-robot interaction
2011
Earlier work this paper cites.

Similar

H. Larochelle and I. Murray, “The neural autoregressive distribution estimator.,” in AISTATS
2011
Cited alongside, same era.
T. Tieleman and G. Hinton, “Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude,” COURSERA: Neural Networks for Machine Learning
2012
Cited alongside, same era.
B. Kehoe, A. Matsukawa, S. Candido, J. Kuffner, and K. Goldberg, “Cloud-based robot grasping with the Google object recognition engine,” in IEEE International Conference on Robotics and Automation (ICRA)
2013
Cited alongside, same era.
J. Kober, J. A. Bagnell, and J. Peters, “Reinforcement learning in robotics: A survey,” The International Journal of Robotics Research
2013
Cited alongside, same era.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114
Original
2013
Cited alongside, same era.
A. Graves, “Generating sequences with recurrent neural networks,” arXiv preprint arXiv:1308.0850
Original
2013
Cited alongside, same era.
M. Forbes, M. J.-Y. Chung, M. Cakmak, and R. P. Rao, “Robot programming by demonstration with crowdsourced action fixes,” in Second AAAI Conference on Human Computation and Crowdsourcing
2014
Cited alongside, same era.
M. P. Deisenroth, P. Englert, J. Peters, and D. Fox, “Multi-task policy search for robotics,” in IEEE International Conference on Robotics and Automation (ICRA)
2014
Cited alongside, same era.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems
2014
Cited alongside, same era.
D. P. Kingma, S. Mohamed, D. J. Rezende, and M. Welling, “Semi-supervised learning with deep generative models,” in Advances in Neural Information Processing Systems
2014
Cited alongside, same era.
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” The Journal of Machine Learning Research
2014
Cited alongside, same era.
I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Advances in neural information processing systems (NIPS)
2014
Cited alongside, same era.

Then

T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971
Original
2015
Later among the works it cites.
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167
Original
2015
Later among the works it cites.
M. Kopicki, R. Detry, M. Adjigble, R. Stolkin, A. Leonardis, and J. L. Wyatt, “One-shot learning and generation of dexterous grasps for novel objects,” The International Journal of Robotics Research
2016
Later among the works it cites.
R. Lioutikov, O. Kroemer, G. Maeda, and J. Peters, “Learning manipulation by sequencing motor primitives with a two-armed robot,” in Intelligent Autonomous Systems
2016
Later among the works it cites.
Z. Fang, G. Bartels, and M. Beetz, “Learning models for constraint-based motion parameterization from interactive physics-based simulation,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
2016
Later among the works it cites.
S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” Journal of Machine Learning Research
2016
Later among the works it cites.
L. Pinto and A. Gupta, “Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours,” in IEEE International Conference on Robotics and Automation (ICRA)
2016
Later among the works it cites.
P. Agrawal, A. Nair, P. Abbeel, J. Malik, and S. Levine, “Learning to poke by poking: Experiential learning of intuitive physics,” arXiv preprint arXiv:1606.07419
Original
2016
Later among the works it cites.
S. Levine, P. Pastor, A. Krizhevsky, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection,” arXiv preprint arXiv:1603.02199
Original
2016
Later among the works it cites.
C. Devin, A. Gupta, T. Darrell, P. Abbeel, and S. Levine, “Learning modular neural network policies for multi-task and multi-robot transfer,” arXiv preprint arXiv:1609.07088
Original
2016
Later among the works it cites.
L. Pinto and A. Gupta, “Learning to push by grasping: Using multiple tasks for effective learning,” arXiv preprint arXiv:1609.09025
Original
2016
Later among the works it cites.
R. Rahmatizadeh, P. Abolghasemi, A. Behal, and L. Bölöni, “Learning real manipulation tasks from virtual demonstrations using LSTM,” arXiv preprint arXiv:1603.03833
Original
2016
Later among the works it cites.
A. B. L. Larsen, S. K. Sønderby, H. Larochelle, and O. Winther, “Autoencoding beyond pixels using a learned similarity metric,” in International Conference on Machine Learning
2016
Later among the works it cites.
J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450
Original
2016
Later among the works it cites.
A. V. d. Oord, N. Kalchbrenner, and K. Kavukcuoglu, “Pixel recurrent neural networks,” in International Conference on Machine Learning
2016
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…