Fetching the paper…
Reading the bibliography…
The performance of deep neural networks is highly sensitive to the choice of the hyperparameters that define the structure of the network and the learning process.
“Direct Search” Solution of Numerical and Statistical Problems
R. Hooke and T.A. Jeeves · 1961
Earlier work this paper cites.
On the convergence of pattern search algorithms
V. Torczon · 1997
Earlier work this paper cites.
Pattern search algorithms for mixed variable programming
C. Audet and J.E. Dennis, Jr · 2001
Earlier work this paper cites.
Mixed variable optimization of the number and composition of heat intercepts in a thermal insulation system
M. Kokkolaras, C. Audet, and J.E. Dennis, Jr · 2001
Earlier work this paper cites.
Mixed variable optimization of a Load-Bearing thermal insulation system using a filter pattern search algorithm
M.A. Abramson · 2004
Earlier work this paper cites.
Mesh Adaptive Direct Search Algorithms for Constrained Optimization
C. Audet and J.E. Dennis, Jr · 2006
Earlier work this paper cites.
Finding optimal algorithmic parameters using derivative-free optimization
C. Audet and D. Orban · 2006
Earlier work this paper cites.
Filter pattern search algorithms for mixed variable constrained optimization problems
M.A. Abramson, C. Audet, and J.E. Dennis, Jr · 2007
Earlier work this paper cites.
Nonsmooth optimization through Mesh Adaptive Direct Search and Variable Neighborhood Search
C. Audet, V. Béchard, and S. Le Digabel · 2008
Earlier work this paper cites.
Mesh Adaptive Direct Search Algorithms for Mixed Variable Optimization
M.A. Abramson, C. Audet, J.W. Chrissis, and J.G. Walston · 2009
Earlier work this paper cites.
Introduction to Derivative-Free Optimization
A.R. Conn, K. Scheinberg, and L.N. Vicente · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
A. Krizhevsky and G. Hinton · 2009
Earlier work this paper cites.
The BOBYQA algorithm for bound constrained optimization without derivatives
M.J.D. Powell · 2009
Earlier work this paper cites.
MNIST handwritten digit database
Y. LeCun and C. Cortes · 2010
Earlier work this paper cites.
Algorithms for hyper-parameter optimization
J. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl · 2011
Earlier work this paper cites.
Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
J. Duchi, E. Hazan, and Y. Singer · 2011
Earlier work this paper cites.
Sequential model-based optimization for general algorithm configuration
F. Hutter, H. H. Hoos, and K. Leyton-Brown · 2011
Earlier work this paper cites.
Algorithm 909: NOMAD: Nonlinear Optimization with the MADS algorithm
S. Le Digabel · 2011
Earlier work this paper cites.
Scikit-learn: Machine Learning in Python
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay · 2011
Earlier work this paper cites.
Practical recommendations for gradient-based training of deep architectures
Y. Bengio · 2012
Earlier work this paper cites.
Random search for hyper-parameter optimization
J. Bergstra and Y. Bengio · 2012
Earlier work this paper cites.
Stochastic Gradient Descent Tricks
L. Bottou · 2012
Cited alongside, same era.
Efficient BackProp
Y.A. LeCun, L. Bottou, G.B. Orr, and K.R. Müller · 2012
Cited alongside, same era.
Practical Bayesian optimization of machine learning algorithms
J. Snoek, H. Larochelle, and R. Prescott Adams · 2012
Cited alongside, same era.
Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude
T. Tieleman and G. Hinton · 2012
Cited alongside, same era.
Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures
J. Bergstra, D. Yamins, and D.D. Cox · 2013
Cited alongside, same era.
Optimization of algorithms with OPAL
C. Audet, C.-K. Dang, and D. Orban · 2014
Cited alongside, same era.
Particle swarm optimization for hyper-parameter selection in deep neural networks
P.R. Lorenzo, J. Nalepa, M. Kawulok, L.S. Ramos, and J.R. Pastor · 2017
Later among the works it cites.
Automatic differentiation in PyTorch
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer · 2017
Later among the works it cites.
BFO, A Trainable Derivative-free Brute Force Optimizer for Nonlinear Bound-constrained Optimization and Equilibrium Computations with Continuous and Discrete Variables
M. Porcelli and Ph.L. Toint · 2017
Later among the works it cites.
A genetic programming approach to designing convolutional neural network architectures
M. Suganuma, S. Shirakawa, and T. Nagao · 2017
Later among the works it cites.
Mesh-based Nelder-Mead algorithm for inequality constrained optimization
C. Audet and C. Tribes · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Caffe: Convolutional architecture for fast feature embedding
Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell · 2014
Cited alongside, same era.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2014
Cited alongside, same era.
Metric Optimization Engine
Yelp · 2014
Cited alongside, same era.
Adam: A Method for Stochastic Optimization
D.P. Kingma and L.B. Jimmy · 2015
Cited alongside, same era.
Optimizing deep learning hyper-parameters through an evolutionary algorithm
S.R. Young, D.C. Rose, T.P. Karnowski, S.H. Lim, and R.M. Patton · 2015
Cited alongside, same era.
Designing neural network architectures using reinforcement learning
B. Baker, O. Gupta, N. Naik, and R. Raskar · 2016
Cited alongside, same era.
DeepHyper: Asynchronous Hyperparameter Search for Deep Neural Networks
P. Balaprakash, M. Salim, T. Uram, V. Vishwanath, and S. Wild · 2018
Later among the works it cites.
Neural architecture search: A survey
T. Elsken, J. H. Metzen, and F. Hutter · 2018
Later among the works it cites.
Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection
S. Levine, P. Pastor, A. Krizhevsky, J. Ibarz, and D. Quillen · 2018
Later among the works it cites.
Hyperband: A novel bandit-based approach to hyperparameter optimization
L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar · 2018
Later among the works it cites.
Tuning BARON using derivative-free optimization algorithms
J. Liu, N. Ploskas, and N.V. Sahinidis · 2018
Later among the works it cites.
Regularized evolution for image classifier architecture search
E. Real, A. Aggarwal, Y. Huang, and Q. V. Le · 2018
Later among the works it cites.
Scalable Gaussian process-based transfer surrogates for hyperparameter optimization
M. Wistuba, N. Schilling, and L. Schmidt-Thieme · 2018
Later among the works it cites.
Towards automated deep learning: Efficient joint neural architecture and hyperparameter search
A. Zela, A. Klein, and S. Falknerand F. Hutter · 2018
Later among the works it cites.
Learning transferable architectures for scalable image recognition
B. Zoph, V. Vasudevan, J. Shlens, and Q.V. Le · 2018
Later among the works it cites.
The Mesh Adaptive Direct Search Algorithm for Granular and Discrete Variables
C. Audet, S. Le Digabel, and C. Tribes · 2019
Closest in time.
Oríon: Asynchronous Distributed Hyperparameter Optimization
X. Bouthillier and C. Tsirigotis · 2019
Closest in time.
A Beginner’s Guide To Understanding Convolutional Neural Networks
A. Deshpande · 2019
Closest in time.
Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution
T. Elsken, J. H. Metzen, and F. Hutter · 2019
Closest in time.
VGG16 : Convolutional Network for Classification and Detection
M. Hassan · 2019
Closest in time.
A Novel Orthogonal Direction Mesh Adaptive Direct Search Approach for SVM Hyperparameter Tuning
A.R. Mello, J. de Matos, M.R. Stemmer, A. de Souza Britto Jr, and A.L. Koerich · 2019
Closest in time.
Introduction To Convolutional Neural Networks
V. Pavlovsky · 2019
Closest in time.