2017

The Intentional Unintentional Agent: Learning to Solve Many Continuous Control Tasks Simultaneously

Cabi, Serkan, Colmenarejo, Sergio Gómez, Hoffman, Matthew W. et al.

Understand

This paper introduces the Intentional Unintentional (IU) agent.

This agent endows the deep deterministic policy gradients (DDPG) agent for continuous control with the ability to solve several tasks simultaneously.
Learning to solve many tasks simultaneously has been a long-standing, core goal of artificial intelligence, inspired by infant development and motivated by the desire to build flexible robot manipulators capable of many diverse behaviours.
We show that the IU agent not only learns to solve many tasks simultaneously but it also learns faster than agents that target a single task at-a-time.

Built on

Rhythmical stereotypies in normal human infants
E. Thelen · 1979
Earlier work this paper cites.
Intelligence without representation
R. A. Brooks · 1991
Earlier work this paper cites.
The MAXQ method for hierarchical reinforcement learning
T. G. Dietterich · 1998
Earlier work this paper cites.
Intra-option learning about temporally abstract actions
R. S. Sutton, D. Precup, and S. P. Singh · 1998
Earlier work this paper cites.
Fetal and neonatal hand movement
J. W. Sparling, J. Van Tol, and N. C. Chescheir · 1999
Earlier work this paper cites.
Policy gradient methods for reinforcement learning with function approximation
R. S. Sutton, D. A. McAllester, S. P. Singh, and Y. Mansour · 1999
Earlier work this paper cites.
Q-decomposition for reinforcement learning agents
S. Russell and A. Zimdars · 2003
Earlier work this paper cites.
Developmental robotics: a survey
M. Lungarella, G. Metta, R. Pfeifer, and G. Sandini · 2003
Earlier work this paper cites.
Curriculum learning
Y. Bengio, J. Louradour, R. Collobert, and J. Weston · 2009
Earlier work this paper cites.
Horde: A scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction
R. S. Sutton, J. Modayil, M. Delp, T. Degris, P. M. Pilarski, A. White, and D. Precup · 2011
Earlier work this paper cites.
Mujoco: A physics engine for model-based control
E. Todorov, T. Erez, and Y. Tassa · 2012
Earlier work this paper cites.
A survey of multi-objective sequential decision-making
D. M. Roijers, P. Vamplew, S. Whiteson, and R. Dazeley · 2013
Earlier work this paper cites.

Similar

Twitching in sensorimotor development from sleeping rats to robots
M. S. Blumberg, H. G. Marques, and F. Iida · 2013
Cited alongside, same era.
Deterministic policy gradient algorithms
D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller · 2014
Cited alongside, same era.
Learning to execute
Original
W. Zaremba and I. Sutskever · 2014
Cited alongside, same era.
Motor development
K. E. Adolph and S. R. Robinson · 2015
Cited alongside, same era.
Universal value function approximators
T. Schaul, D. Horgan, K. Gregor, and D. Silver · 2015
Cited alongside, same era.
Human-level control through deep reinforcement learning
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis · 2015
Cited alongside, same era.
Recurrent reinforcement learning: a hybrid approach
Original
X. Li, L. Li, J. Gao, X. He, J. Chen, L. Deng, and J. He · 2015
Cited alongside, same era.
Adam: A method for stochastic optimization
D. Kingma and J. Ba · 2015
Cited alongside, same era.
Continuous control with deep reinforcement learning
T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra · 2016
Cited alongside, same era.
Asynchronous methods for deep reinforcement learning
V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu · 2016
Cited alongside, same era.
Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation
T. D. Kulkarni, K. Narasimhan, A. Saeedi, and J. Tenenbaum · 2016
Cited alongside, same era.

Then

Playing FPS games with deep reinforcement learning
Original
G. Lample and D. S. Chaplot · 2016
Later among the works it cites.
Neural programmer-interpreters
S. Reed and N. de Freitas · 2016
Later among the works it cites.
Learning the curriculum with Bayesian optimization for task-specific word representation learning
Y. Tsvetkov, M. Faruqui, W. Ling, B. MacWhinney, and C. Dyer · 2016
Later among the works it cites.
Reinforcement learning with unsupervised auxiliary tasks
M. Jaderberg, V. Mnih, W. M. Czarnecki, T. Schaul, J. Z. Leibo, D. Silver, and K. Kavukcuoglu · 2017
Closest in time.
Hybrid reward architecture for reinforcement learning
H. van Seijen, M. Fatemi, J. Romoff, R. Laroche, T. Barnes, and J. Tsang · 2017
Closest in time.
Learning to act by predicting the future
A. Dosovitskiy and V. Koltun · 2017
Closest in time.
Automated curriculum learning for neural networks
Original
A. Graves, M. G. Bellemare, J. Menick, R. Munos, and K. Kavukcuoglu · 2017
Closest in time.
Environment-independent task specifications via GLTL
M. L. Littman, U. Topcu, J. Fu, C. Isbell, M. Wen, and J. MacGlashan · 2017
Closest in time.
A deep compositional framework for human-like language acquisition in virtual environment
H. Yu, H. Zhang, and W. Xu · 2017
Closest in time.
Grounded language learning in a simulated 3D world
K. M. Hermann, F. Hill, S. Green, F. Wang, R. Faulkner, H. Soyer, D. Szepesvari, W. Czarnecki, M. Jaderberg, D. Teplyashin, M. Wainwright, C. Apps, D. Hassabis, and P. Blunsom · 2017
Closest in time.
Programmable agents
M. Denil, S. G. Colmenarejo, S. Cabi, D. Saxton, and N. de Freitas · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…