2020

Training Stronger Baselines for Learning to Optimize

Chen, Tianlong, Zhang, Weiyi, Zhou, Jingyang et al.

Understand

Learning to optimize (L2O) has gained increasing attention since classical optimizers require laborious problem-specific design and hyperparameter tuning.

However, there is a gap between the practical demand and the achievable performance of existing L2O models.
Specifically, those learned optimizers are applicable to only a limited class of problems, and often exhibit instability.
With many efforts devoted to designing more sophisticated L2O models, we argue for another orthogonal, under-explored theme: the training techniques for those L2O models.

Built on

Genetic algorithms and machine learning
David E Goldberg and John Henry Holland · 1988
Earlier work this paper cites.
Learning a synaptic learning rule
Yoshua Bengio, Samy Bengio, and Jocelyn Cloutier · 1990
Earlier work this paper cites.
On the optimization of a synaptic learning rule
Samy Bengio, Yoshua Bengio, Jocelyn Cloutier, and Jan Gecsei · 1992
Earlier work this paper cites.
Self-improving reactive agents based on reinforcement learning, planning and teaching
Long-Ji Lin · 1992
Earlier work this paper cites.
On the search for new learning rules for ANNs
Samy Bengio, Yoshua Bengio, and Jocelyn Cloutier · 1995
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
Fixed-weight on-line learning
A Steven Younger, Peter R Conwell, and Neil E Cotter · 1999
Earlier work this paper cites.
Is imitation learning the route to humanoid robots?
Stefan Schaal · 1999
Earlier work this paper cites.
Learning to learn using gradient descent
Sepp Hochreiter, A Steven Younger, and Peter R Conwell · 2001
Earlier work this paper cites.
Computational approaches to motor learning by imitation
Stefan Schaal, Auke Ijspeert, and Aude Billard · 2003
Earlier work this paper cites.
Exploration and exploitation in reinforcement learning
Melanie Coggan · 2004
Earlier work this paper cites.
Curriculum learning
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston · 2009
Earlier work this paper cites.
Self-paced learning for latent variable models
M Pawan Kumar, Benjamin Packer, and Daphne Koller · 2010
Earlier work this paper cites.
Toward off-policy learning control with function approximation
Hamid Reza Maei, Csaba Szepesvári, Shalabh Bhatnagar, and Richard S Sutton · 2010
Earlier work this paper cites.
Random search for hyper-parameter optimization
James Bergstra and Yoshua Bengio · 2012
Earlier work this paper cites.

Similar

Practical bayesian optimization of machine learning algorithms
Jasper Snoek, Hugo Larochelle, and Ryan P Adams · 2012
Cited alongside, same era.
On the difficulty of training recurrent neural networks
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio · 2013
Cited alongside, same era.
Learning to execute
Original
Wojciech Zaremba and Ilya Sutskever · 2014
Cited alongside, same era.
Adam: A method for stochastic optimization
Original
Diederik P Kingma and Jimmy Ba · 2014
Cited alongside, same era.
Self-paced curriculum learning
Lu Jiang, Deyu Meng, Qian Zhao, Shiguang Shan, and Alexander G Hauptmann · 2015
Cited alongside, same era.
Scheduled sampling for sequence prediction with recurrent neural networks
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer · 2015
Cited alongside, same era.
Learning to learn by gradient descent by gradient descent
Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas · 2016
Cited alongside, same era.
Learning to optimize
Original
Ke Li and Jitendra Malik · 2016
Cited alongside, same era.
Learning to learn without gradient descent by gradient descent
Yutian Chen, Matthew W Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P Lillicrap, Matt Botvinick, and Nando de Freitas · 2017
Cited alongside, same era.
Learning gradient descent: Better generalization and longer horizons
Kaifeng Lv, Shunhua Jiang, and Jian Li · 2017
Cited alongside, same era.
Learned optimizers that scale and generalize
Olga Wichrowska, Niru Maheswaranathan, Matthew W Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Nando de Freitas, and Jascha Sohl-Dickstein · 2017
Cited alongside, same era.
Neural optimizer search with reinforcement learning
Irwan Bello, Barret Zoph, Vijay Vasudevan, and Quoc V Le · 2017
Cited alongside, same era.

Then

Reverse curriculum generation for reinforcement learning
Original
Carlos Florensa, David Held, Markus Wulfmeier, Michael Zhang, and Pieter Abbeel · 2017
Later among the works it cites.
Understanding and correcting pathologies in the training of learned optimizers
Original
Luke Metz, Niru Maheswaranathan, Jeremy Nixon, C Daniel Freeman, and Jascha Sohl-Dickstein · 2018
Later among the works it cites.
Learning to optimize in swarms
Yue Cao, Tianlong Chen, Zhangyang Wang, and Yang Shen · 2019
Later among the works it cites.
Pipps: Flexible model-based policy search robust to the curse of chaos
Original
Paavo Parmas, Carl Edward Rasmussen, Jan Peters, and Kenji Doya · 2019
Later among the works it cites.
Rna secondary structure prediction by learning unrolled algorithms
Xinshi Chen, Yu Li, Ramzan Umarov, Xin Gao, and Le Song · 2019
Later among the works it cites.
Lookahead optimizer: k steps forward, 1 step back
Michael Zhang, James Lucas, Jimmy Ba, and Geoffrey E Hinton · 2019
Later among the works it cites.
Searching for low-bit weights in quantized neural networks
Original
Zhaohui Yang, Yunhe Wang, Kai Han, Chunjing Xu, Chao Xu, Dacheng Tao, and Chang Xu · 2020
Closest in time.
Learning to stop while learning to predict
Original
Xinshi Chen, Hanjun Dai, Yu Li, Xin Gao, and Le Song · 2020
Closest in time.
Halo: Hardware-aware learning to optimize
Chaojian Li, Tianlong Chen, Haoran You, Zhangyang Wang, and Yingyan Lin · 2020
Closest in time.
L2-gcn: Layer-wise and learned efficient training of graph convolutional networks
Yuning You, Tianlong Chen, Zhangyang Wang, and Yang Shen · 2020
Closest in time.
Automated synthetic-to-real generalization
Wuyang Chen, Zhiding Yu, Zhangyang Wang, and Anima Anandkumar · 2020
Closest in time.
Self-pu: Self boosted and calibrated positive-unlabeled training
Xuxi Chen, Wuyang Chen, Tianlong Chen, Ye Yuan, Chen Gong, Kewei Chen, and Zhangyang Wang · 2020
Closest in time.
Learning to plan in high dimensions via neural exploration-exploitation trees
Binghong Chen, Bo Dai, Qinjie Lin, Guo Ye, Han Liu, and Le Song · 2020
Closest in time.
Nas-bench-201: Extending the scope of reproducible neural architecture search
Original
Xuanyi Dong and Yi Yang · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…