2017

Teacher-Student Curriculum Learning

Matiisen, Tambet, Oliver, Avital, Cohen, Taco et al.

Understand

We propose Teacher-Student Curriculum Learning (TSCL), a framework for automatic curriculum learning, where the Student tries to learn a complex task and the Teacher automatically chooses subtasks from a given set for the Student to train on.

We describe a family of Teacher algorithms that rely on the intuition that the Student should practice more those tasks on which it makes the fastest progress, i.e.
where the slope of the learning curve is highest.
In addition, the Teacher algorithms address the problem of forgetting by also choosing tasks where the Student's performance is getting worse.

Built on

Curious model-building control systems
Jürgen Schmidhuber · 1991
Earlier work this paper cites.
Reinforcement learning: An introduction
Richard S. Sutton and Andrew G. Barto · 1998
Earlier work this paper cites.
The non-stochastic multi-armed bandit problem
Peter Auer, Nicolò‚ Cesa-Bianchi, Yoav Freund, and Robert E. Schapire · 2002
Earlier work this paper cites.
What is intrinsic motivation? a typology of computational approaches
Pierre-Yves Oudeyer and Frederic Kaplan · 2007
Earlier work this paper cites.
Intrinsic motivation systems for autonomous mental development
Pierre-Yves Oudeyer, Frdric Kaplan, and Verena V Hafner · 2007
Earlier work this paper cites.
Curriculum learning
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston · 2009
Earlier work this paper cites.
Formal theory of creativity, fun, and intrinsic motivation (1990–2010)
Jürgen Schmidhuber · 2010
Earlier work this paper cites.
Efficient exploration in reinforcement learning
John Langford · 2011
Earlier work this paper cites.
Active learning of inverse models with intrinsically motivated goal exploration in robots
Adrien Baranes and Pierre-Yves Oudeyer · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
Diederik Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Sequence to sequence learning with neural networks
Ilya Sutskever, Oriol Vinyals, and Quoc V Le · 2014
Earlier work this paper cites.
Learning to execute., 2014
Original
Wojciech Zaremba and Ilya Sutskever · 2014
Earlier work this paper cites.

Similar

Multi-armed bandits for intelligent tutoring systems
Benjamin Clement, Didier Roy, Pierre-Yves Oudeyer, and Manuel Lopes · 2015
Cited alongside, same era.
Neural gpus learn algorithms
Original
Łukasz Kaiser and Ilya Sutskever · 2015
Cited alongside, same era.
Grid long short-term memory
Original
Nal Kalchbrenner, Ivo Danihelka, and Alex Graves · 2015
Cited alongside, same era.
End-to-end training of deep visuomotor policies, 2015
Original
Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel · 2015
Cited alongside, same era.
Continuous control with deep reinforcement learning
Original
Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra · 2015
Cited alongside, same era.
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, and et al. Alex Graves · 2015
Cited alongside, same era.
Neural programmer-interpreters
Original
Scott Reed and Nando De Freitas · 2015
Cited alongside, same era.
Trust region policy optimization
John Schulman, Sergey Levine, Pieter Abbeel, Michael I Jordan, and Philipp Moritz · 2015
Cited alongside, same era.
Incentivizing exploration in reinforcement learning with deep predictive models
Original
Bradly C Stadie, Sergey Levine, and Pieter Abbeel · 2015
Cited alongside, same era.
Reinforcement learning through asynchronous advantage actor-critic on a gpu
Mohammad Babaeizadeh, Iuri Frosio, Stephen Tyree, Jason Clemons, and Jan Kautz · 2016
Cited alongside, same era.
Unifying count-based exploration and intrinsic motivation
Marc Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos · 2016
Cited alongside, same era.
Hybrid computing using a neural network with dynamic external memory
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, and et al. Sergio Gómez Colmenarejo · 2016
Cited alongside, same era.

Then

Vime: Variational information maximizing exploration
Rein Houthooft, Xi Chen, Yan Duan, John Schulman, Filip De Turck, and Pieter Abbeel · 2016
Later among the works it cites.
The malmo platform for artificial intelligence experimentation
Matthew Johnson, Katja Hofmann, Tim Hutton, and David Bignell · 2016
Later among the works it cites.
Learning to navigate in complex environments
Original
Piotr Mirowski, Razvan Pascanu, Fabio Viola, Hubert Soyer, Andy Ballard, Andrea Banino, Misha Denil, Ross Goroshin, Laurent Sifre, Koray Kavukcuoglu, et al · 2016
Later among the works it cites.
Control of memory, active perception, and action in minecraft
Original
Junhyuk Oh, Valliappa Chockalingam, Satinder Singh, and Honglak Lee · 2016
Later among the works it cites.
A deep hierarchical approach to lifelong learning in minecraft
Original
Chen Tessler, Shahar Givony, Tom Zahavy, Daniel J Mankowitz, and Shie Mannor · 2016
Later among the works it cites.
Automated curriculum learning for neural networks, 2017
Original
Alex Graves, Marc G. Bellemare, Jacob Menick, Remi Munos, and Koray Kavukcuoglu · 2017
Closest in time.
Automatic goal generation for reinforcement learning agents
Original
David Held, Xinyang Geng, Carlos Florensa, and Pieter Abbeel · 2017
Closest in time.
Faster reinforcement learning using active simulators
Original
Vikas Jain and Theja Tulabandhula · 2017
Closest in time.
Proximal policy optimization algorithms
Original
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov · 2017
Closest in time.
Online multi-task learning using biased sampling
Original
Sahil Sharma and Balaraman Ravindran · 2017
Closest in time.
Intrinsic motivation and automatic curricula via asymmetric self-play
Original
Sainbayar Sukhbaatar, Ilya Kostrikov, Arthur Szlam, and Rob Fergus · 2017
Closest in time.
Training agent for first-person shooter game with actor-critic curriculum learning
Yuxin Wu and Yuandong Tian · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…