Learning Longer Memory in Recurrent Neural Networks

Learning internal representations by error propagation

Rumelhart, David E, Hinton, Geoffrey E, and Williams, Ronald J · 1985

Earlier work this paper cites.

Attractor dynamics and parallelism in a connectionist sequential machine

Jordan, Michael I · 1987

Earlier work this paper cites.

Generalization of backpropagation with application to a recurrent gas market model

Werbos, Paul J · 1988

Earlier work this paper cites.

A focused back-propagation algorithm for temporal pattern recognition

Mozer, Michael C · 1989

Earlier work this paper cites.

Finding structure in time

Elman, Jeffrey L · 1990

Earlier work this paper cites.

A cache-based natural language model for speech recognition

Kuhn, Roland and De Mori, Renato · 1990

Earlier work this paper cites.

Neural net architectures for temporal sequence processing

Mozer, Michael C · 1993

Earlier work this paper cites.

Learning long-term dependencies with gradient descent is difficult

Bengio, Yoshua, Simard, Patrice, and Frasconi, Paolo · 1994

Earlier work this paper cites.

Gradient-based learning algorithms for recurrent networks and their computational complexity

Williams, Ronald J and Zipser, David · 1995

Earlier work this paper cites.

Long short-term memory

Hochreiter, Sepp and Schmidhuber, Jürgen · 1997

Earlier work this paper cites.

The HTK book , volume 2

Young, Steve, Evermann, Gunnar, Gales, Mark, Hain, Thomas, Kershaw, Dan, Liu, Xunying, Moore, Gareth, Odell, Julian, Ollason, Dave, Povey, Dan, et al · 1997

Cited alongside, same era.

The vanishing gradient problem during learning recurrent neural nets and problem solutions

Hochreiter, Sepp · 1998

Cited alongside, same era.

Efficient backprop

LeCun, Yann, Bottou, Leon, Orr, Genevieve, and Müller, Klaus · 1998

Cited alongside, same era.

Classes for fast maximum entropy training

Goodman, Joshua · 2001

Cited alongside, same era.

Framewise phoneme classification with bidirectional lstm and other neural network architectures

Graves, Alex and Schmidhuber, Jürgen · 2005

Cited alongside, same era.

Optimization and applications of echo state networks with leaky-integrator neurons

Jaeger, Herbert, Lukoševičius, Mantas, Popovici, Dan, and Siewert, Udo · 2007

Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition

Dahl, George E, Yu, Dong, Deng, Li, and Acero, Alex · 2012

Later among the works it cites.

Statistical language models based on neural networks

Mikolov, Tomáš · 2012

Later among the works it cites.

Context dependent recurrent neural network language model

Mikolov, Tomas and Zweig, Geoffrey · 2012

Later among the works it cites.

Lstm neural networks for language modeling

Sundermeyer, Martin, Schlüter, Ralf, and Ney, Hermann · 2012

Later among the works it cites.

Advances in optimizing recurrent networks

Bengio, Yoshua, Boulanger-Lewandowski, Nicolas, and Pascanu, Razvan · 2013

Later among the works it cites.

Regularization and nonlinearities for neural language models: when are they needed?

Original