Hyperparameter optimization with REINFORCE and Transformers

V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” in International conference on machine learning , 2016, pp. 1928–1937

1937

Earlier work this paper cites.

R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforcement learning,” Machine learning , vol. 8, no. 3-4, pp. 229–256, 1992

1992

Earlier work this paper cites.

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation , vol. 9, no. 8, pp. 1735–1780, 1997

1997

Earlier work this paper cites.

V. R. Konda and J. N. Tsitsiklis, “Actor-critic algorithms,” in Advances in neural information processing systems , 2000, pp. 1008–1014

2000

Earlier work this paper cites.

C. K. Williams and C. E. Rasmussen, Gaussian processes for machine learning . MIT press Cambridge, MA, 2006, vol. 2, no. 3

2006

Earlier work this paper cites.

S. Bickel, M. Brückner, and T. Scheffer, “Discriminative learning under covariate shift.” Journal of Machine Learning Research , vol. 10, no. 9, 2009

2009

Earlier work this paper cites.

T. Mikolov, M. Karafiát, L. Burget, J. Černockỳ, and S. Khudanpur, “Recurrent neural network based language model,” in Eleventh annual conference of the international speech communication association , 2010

2010

Earlier work this paper cites.

J. S. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl, “Algorithms for hyper-parameter optimization,” in Advances in neural information processing systems , 2011, pp. 2546–2554

2011

Earlier work this paper cites.

J. Bergstra and Y. Bengio, “Random search for hyper-parameter optimization,” Journal of machine learning research , vol. 13, no. Feb, pp. 281–305, 2012

2012

Earlier work this paper cites.

J. Snoek, H. Larochelle, and R. P. Adams, “Practical bayesian optimization of machine learning algorithms,” in Advances in neural information processing systems , 2012, pp. 2951–2959

2012

Earlier work this paper cites.

C. Thornton, F. Hutter, H. H. Hoos, and K. Leyton-Brown, “Auto-weka: Combined selection and hyperparameter optimization of classification algorithms,” in Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining , 2013, pp. 847–855

2013

Earlier work this paper cites.

R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty of training recurrent neural networks,” in International conference on machine learning , 2013, pp. 1310–1318

2013

Earlier work this paper cites.

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems , 2014, pp. 2672–2680

2014

Earlier work this paper cites.

D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473 , 2014

Original

2014

Earlier work this paper cites.

D. Maclaurin, D. Duvenaud, and R. Adams, “Gradient-based hyperparameter optimization through reversible learning,” in International Conference on Machine Learning , 2015, pp. 2113–2122

2015

Earlier work this paper cites.

M. Germain, K. Gregor, I. Murray, and H. Larochelle, “Made: Masked autoencoder for distribution estimation,” in International Conference on Machine Learning , 2015, pp. 881–889

2015

Earlier work this paper cites.

J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High-dimensional continuous control using generalized advantage estimation,” arXiv preprint arXiv:1506.02438 , 2015

Original

2015

Earlier work this paper cites.

D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” in Proceedings of the International Conference on Learning Representations , 2015

2015

Earlier work this paper cites.

D. Harrison and D. Rubinfeld, “Boston housing dataset,” 2015

2015

Earlier work this paper cites.

J. Bergstra, B. Komer, C. Eliasmith, D. Yamins, and D. D. Cox, “Hyperopt: a python library for model selection and hyperparameter optimization,” Computational Science & Discovery , vol. 8, no. 1, p. 014008, 2015

2015

Earlier work this paper cites.

B. Zoph and Q. V. Le, “Neural architecture search with reinforcement learning,” arXiv preprint arXiv:1611.01578 , 2016

Original

2016

Cited alongside, same era.

B. Baker, O. Gupta, N. Naik, and R. Raskar, “Designing neural network architectures using reinforcement learning,” arXiv preprint arXiv:1611.02167 , 2016

Original

2016

Cited alongside, same era.

Y. Kim, Y. Jernite, D. Sontag, and A. M. Rush, “Character-aware neural language models,” in Thirtieth AAAI Conference on Artificial Intelligence , 2016

2016

Cited alongside, same era.

A. v. d. Oord, N. Kalchbrenner, and K. Kavukcuoglu, “Pixel recurrent neural networks,” arXiv preprint arXiv:1601.06759 , 2016

Original

2016

Cited alongside, same era.

A. v. d. Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,” arXiv preprint arXiv:1609.03499 , 2016

S. Falkner, A. Klein, and F. Hutter, “Bohb: Robust and efficient hyperparameter optimization at scale,” arXiv preprint arXiv:1807.01774 , 2018

Original

2018

Later among the works it cites.

N. Fusi, R. Sheth, and M. Elibol, “Probabilistic matrix factorization for automated machine learning,” in Advances in neural information processing systems , 2018, pp. 3348–3357

2018

Later among the works it cites.

C. Liu, B. Zoph, M. Neumann, J. Shlens, W. Hua, L.-J. Li, L. Fei-Fei, A. Yuille, J. Huang, and K. Murphy, “Progressive neural architecture search,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 19–34

2018

Later among the works it cites.

H. Pham, M. Y. Guan, B. Zoph, Q. V. Le, and J. Dean, “Efficient neural architecture search via parameter sharing,” arXiv preprint arXiv:1802.03268 , 2018

Original

2018

Later among the works it cites.