Fetching the paper…

Reading the bibliography…

2017

Changing Model Behavior at Test-Time Using Reinforcement Learning

Odena, Augustus, Lawson, Dieterich, Olah, Christopher

Original
PDF
Post on X

Understand

Machine learning models are often used at test-time subject to constraints and trade-offs not present at training-time.

For example, a computer vision model operating on an embedded device may need to perform real-time inference, or a translation model operating on a cell phone may wish to bound its average compute time in order to be power-efficient.
In this work we describe a mixture-of-experts model and show how to change its test-time resource-usage on a per-input basis using reinforcement learning.
We test our method on a small MNIST-based example.

Changing Model Behavior at Test-Time Using Reinforcement Learning · Around

Built on

Simple statistical gradient-following algorithms for connectionist reinforcement learning
Ronald J Williams · 1992
Earlier work this paper cites.
Learning factored representations in a deep mixture of experts
Original
David Eigen, Marc’Aurelio Ranzato, and Ilya Sutskever · 2013
Earlier work this paper cites.
Deep sequential neural network
Original
Ludovic Denoyer and Patrick Gallinari · 2014
Earlier work this paper cites.

Similar

Conditional computation in neural networks for faster models
Original
Emmanuel Bengio, Pierre-Luc Bacon, Joelle Pineau, and Doina Precup · 2015
Cited alongside, same era.
Learning to compose neural networks for question answering
Original
Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein · 2016
Cited alongside, same era.
Spatially adaptive computation time for residual networks
Original
Michael Figurnov, Maxwell D. Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry P. Vetrov, and Ruslan Salakhutdinov · 2016
Cited alongside, same era.

Then

Adaptive computation time for recurrent neural networks
Original
Alex Graves · 2016
Later among the works it cites.
Pathnet: Evolution channels gradient descent in super neural networks
Original
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, and Daan Wierstra · 2017
Closest in time.
Outrageously large neural networks: The sparsely-gated mixture-of-experts layer
Original
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, and Jeff Dean · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…