Fast Approximation of Rotations and Hessians matrices

Updating quasi-newton matrices with limited storage

Jorge Nocedal · 1980

Earlier work this paper cites.

Improving the convergence of back-propagation learning with second-order methods

S. Becker and Y. LeCun · 1989

Earlier work this paper cites.

Automatic learning rate maximization by on-line estimation of the hessian’s eigenvectors

Y. LeCun, P. Simard, and B. Pearlmutter · 1993

Earlier work this paper cites.

Methods for generating random orthogonal matrices

A. Genz · 1998

Earlier work this paper cites.

Efficient backprop

Y. LeCun, L. Bottou, G. Orr, and K. Muller · 1998

Earlier work this paper cites.

A stochastic quasi-newton method for online convex optimization

Nicol Schraudolph, Jin Yu, and Simon Günter · 2007

Cited alongside, same era.

Sgd-qn: Careful quasi-newton stochastic gradient descent

Antoine Bordes, Léon Bottou, and Patrick Gallinari · 2009

Cited alongside, same era.

Improved preconditioner for hessian free optimization

Olivier Chapelle and Dumitru Erhan · 2011

Cited alongside, same era.

The sparse matrix transform for covariance estimation and analysis of high dimensional signals

Guangzhi Cao, Leonardo R. Bachega, and Charles A. Bouman · 2011

Cited alongside, same era.

On optimization methods for deep learning

Jiquan Ngiam, Adam Coates, Ahbik Lahiri, Bobby Prochnow, Andrew Ng, and Quoc V Le · 2011

Later among the works it cites.

Large scale distributed deep networks

Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Quoc Le, Mark Mao, Marc’Aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, and Andrew Ng · 2012

Later among the works it cites.

Estimating the hessian by back-propagating curvature

Original

James Martens, Ilya Sutskever, and Kevin Swersky · 2012

Later among the works it cites.

personal communication, 2013

Gérard Ben Arous · 2013

Later among the works it cites.