Recurrent neural network training with preconditioned stochastic gradient descent

P. J. Werbos, “Backpropagation through time: what it does and how to do it,” Proc. IEEE , vol. 78, no. 10, pp. 1550–1560, Oct. 1990

1990

Earlier work this paper cites.

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation , vol. 9, no.8, pp. 1735–1780, 1997

1997

Earlier work this paper cites.

N. Schraudolph, “Fast curvature matrix-vector products for second-order gradient descent,” Neural Computation , vol. 14, no. 7, pp. 1723–1738, 2002

2002

Earlier work this paper cites.

J. Martens and I. Sutskever, “Learning recurrent neural networks with Hessian-free optimization,” In Proc. of the 28th ICML , 2011

2011

Cited alongside, same era.

R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty of training recurrent neural networks,” arXiv:1211.5063, 2012

Original

2012

Cited alongside, same era.

Y. LeCun, C. Cortes, and C. J. C. Burges, THE MNIST DATABASE . Retrieved from http://yann.lecun.com/exdb/mnist/

Original

Cited in the paper.

K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber, “LSTM: a search space odyssey,” arXiv:1503.04069, 2015

Original

2015

Later among the works it cites.

X.-L. Li, “Preconditioned stochastic gradient descent,” arXiv:1512.04202, 2015

Original

2015

Later among the works it cites.