Fetching the paper…

Reading the bibliography…

2017

Bayesian Optimization for Parameter Tuning of the XOR Neural Network

Stewart, Lawrence, Stalzer, Mark

Original
PDF
Post on X

Understand

When applying Machine Learning techniques to problems, one must select model parameters to ensure that the system converges but also does not become stuck at the objective function's local minimum.

Tuning these parameters becomes a non-trivial task for large models and it is not always apparent if the user has found the optimal parameters.
We aim to automate the process of tuning a Neural Network, (where only a limited number of parameter search attempts are available) by implementing Bayesian Optimization.
In particular, by assigning Gaussian Process Priors to the parameter space, we utilize Bayesian Optimization to tune an Artificial Neural Network used to learn the XOR function, with the result of achieving higher prediction accuracy.

Bayesian Optimization for Parameter Tuning of the XOR Neural Network · Around

Built on

Increased rates of convergence through learning rate adaptation
Robert A Jacobs · 1988
Earlier work this paper cites.
Finding structure in time
Jeffrey L Elman · 1990
Earlier work this paper cites.
Application of bayesian approach to numerical methods of global and stochastic optimization
Jonas Mockus · 1994
Earlier work this paper cites.
Regularization theory and neural networks architectures
Federico Girosi, Michael Jones, and Tomaso Poggio · 1995
Earlier work this paper cites.

Similar

Gaussian processes for machine learning
Carl Edward Rasmussen and Christopher KI Williams · 2006
Cited alongside, same era.
A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning
Original
Eric Brochu, Vlad M. Cora, and Nando de Freitas · 2010
Cited alongside, same era.
Fast direct methods for gaussian processes
Original
Sivaram Ambikasaran, Daniel Foreman-Mackey, Leslie Greengard, David W Hogg, and Michael O’Neil · 2014
Cited alongside, same era.
Bayesian optimization with exponential convergence
Kenji Kawaguchi, Leslie Pack Kaelbling, and Tomás Lozano-Pérez · 2015
Cited alongside, same era.

Then

Scalable bayesian optimization using deep neural networks
Original
Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Mostofa Patwary, Mr Prabhat, and Ryan Adams · 2015
Later among the works it cites.
Computer science - theory and applications
Gerhard J. Woeginger Alexander S. Kulikov · 2016
Later among the works it cites.
Bayesian optimization in a billion dimensions via random embeddings
Ziyu Wang, Frank Hutter, Masrour Zoghi, David Matheson, and Nando de Feitas · 2016
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…