Approximating Real-Time Recurrent Learning with Random Kronecker Factors

Learning representations by back-propagating errors

D. E. Rumelhart, G. E. Hinton, and R. J. Williams · 1986

Earlier work this paper cites.

A learning algorithm for continually running fully recurrent neural networks

R. J. Williams and D. Zipser · 1989

Earlier work this paper cites.

An efficient gradient-based algorithm for on-line training of recurrent network trajectories

R. J. Williams and J. Peng · 1990

Earlier work this paper cites.

A method for improving the real-time recurrent learning algorithm

T. Catfolis · 1993

Earlier work this paper cites.

Building a large annotated corpus of english: The penn treebank

M. P. Marcus, M. A. Marcinkiewicz, and B. Santorini · 1993

Earlier work this paper cites.

Learning long-term dependencies with gradient descent is difficult

Y. Bengio, P. Simard, and P. Frasconi · 1994

Earlier work this paper cites.

Long short-term memory

S. Hochreiter and J. Schmidhuber · 1997

Cited alongside, same era.

The “echo state” approach to analysing and training recurrent neural networks-with an erratum note

H. Jaeger · 2001

Cited alongside, same era.

Real-time computing without stable states: A new framework for neural computation based on perturbations

W. Maass, T. Natschläger, and H. Markram · 2002

Cited alongside, same era.

Reservoir computing approaches to recurrent neural network training

M. Lukoševičius and H. Jaeger · 2009

Cited alongside, same era.

Subword language modeling with neural networks

T. Mikolov, I. Sutskever, A. Deoras, H.-S. Le, S. Kombrink, and J. Cernocky · 2012

Cited alongside, same era.

Adam: A method for stochastic optimization

Original

Training recurrent networks online without backtracking

Original

Y. Ollivier, C. Tallec, and G. Charpiat · 2015

Later among the works it cites.

Tensorflow: A system for large-scale machine learning

M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al · 2016

Later among the works it cites.

Decoupled neural interfaces using synthetic gradients

Original

M. Jaderberg, W. M. Czarnecki, S. Osindero, O. Vinyals, A. Graves, D. Silver, and K. Kavukcuoglu · 2016

Later among the works it cites.

Recurrent highway networks

Original

J. G. Zilly, R. K. Srivastava, J. Koutník, and J. Schmidhuber · 2016

Later among the works it cites.

On the state of the art of evaluation in neural language models

Original