Fetching the paper…
Reading the bibliography…
Recurrent neural networks (RNNs) are powerful dynamical models for data with complex temporal structure.
Normierte ringe
I. Gelfand · 1941
Earlier work this paper cites.
Characteristics of random nets of analog neuron-like elements
Shun-Ichi Amari · 1972
Earlier work this paper cites.
Topological and dynamical complexity of random neural networks
Gilles Wainrib and Jonathan Touboul · 1972
Earlier work this paper cites.
Almost sure stable oscillations in a large system for randomly coupled equations
Stuart Geman · 1982
Earlier work this paper cites.
A chaos hypothesis for some large systems of random equations
Stuart Geman and Chii-Ruey Hwang · 1982
Earlier work this paper cites.
Random networks of automata: a simple annealed approximation
Bernard Derrida and Yves Pomeau · 1986
Earlier work this paper cites.
Chaos in random neural networks
H. Sompolinsky, A. Crisanti, and H. J. Sommers · 1988
Earlier work this paper cites.
Learning long-term dependencies with gradient descent is difficult
Yoshua Bengio, Patrice Simard, and Paolo Frasconi · 1994
Earlier work this paper cites.
Increase in Complexity in Random Neural Networks
B Cessac · 1995
Earlier work this paper cites.
Non-hermitian random matrix theory: Method of hermitian reduction
Joshua Feinberg and Anthony Zee · 1997
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Learning to Forget: Continual Prediction with LSTM
Felix A. Gers, J. Schmidhuber, and Fred Cummins · 1999
Earlier work this paper cites.
Statistical properties of eigenvectors in non-hermitian gaussian random matrix ensembles
Bernhard Mehlig and John T Chalker · 2000
Earlier work this paper cites.
Gradient flow in recurrent nets: The difficulty of learning long-term dependencies
Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, and Jürgen Schmidhuber · 2001
Cited alongside, same era.
Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks
Nils Bertschinger and Thomas Natschl · 2004
Cited alongside, same era.
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio · 2010
Cited alongside, same era.
On the importance of initialization and momentum in deep learning
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton · 2012
Cited alongside, same era.
Generating sequences with recurrent neural networks
Alex Graves · 2013
Cited alongside, same era.
On the difficulty of training recurrent neural networks
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
Jeffrey Pennington, Samuel Schoenholz, and Surya Ganguli · 2017
Later among the works it cites.
Deep information propagation
Samuel S Schoenholz, Justin Gilmer, Surya Ganguli, and Jascha Sohl-Dickstein · 2017
Later among the works it cites.
Eigenvalues of non-Hermitian random matrices and Brown measure of non-normal operators: Hermitian reduction and linearization method
Serban T. Belinschi, Piotr Śniady, and Roland Speicher · 2018
Later among the works it cites.
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks
M. Chen, J. Pennington, and S. S. Schoenholz · 2018
Later among the works it cites.
Neural tangent kernel: Convergence and generalization in neural networks
Arthur Jacot, Franck Gabriel, and Clement Hongler · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio · 2013
Cited alongside, same era.
Neural machine translation by jointly learning to align and translate
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio · 2014
Cited alongside, same era.
Learning phrase representations using RNN encoder–decoder for statistical machine translation
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio · 2014
Cited alongside, same era.
Dynamics of random neural networks with bistable units
M Stern, H Sompolinsky, and L F Abbott · 2014
Cited alongside, same era.
Sequence to sequence learning with neural networks
Ilya Sutskever, Oriol Vinyals, and Quoc V Le · 2014
Cited alongside, same era.
Properties of networks with partially structured and partially random connectivity
Yashar Ahmadian, Francesco Fumarola, and Kenneth D Miller · 2015
Cited alongside, same era.
Skip-thought vectors
Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler · 2015
Cited alongside, same era.
Deep Neural Networks as Gaussian Processes
Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S Schoenholz, Jeffrey Pennington, and Jascha Sohl-dickstein · 2018
Later among the works it cites.
Correlations between synapses in pairs of neurons slow down dynamics in randomly connected neural networks
Daniel Marti, Nicolas Brunel, and Srdjan Ostojic · 2018
Later among the works it cites.
Can recurrent neural networks warp time?
Corentin Tallec and Yann Ollivier · 2018
Later among the works it cites.
Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs
Dar Gilboa, Bo Chang, Minmin Chen, Greg Yang, Samuel S Schoenholz, Ed H Chi, and Jeffrey Pennington · 2019
Later among the works it cites.
Gated orthogonal recurrent units: On learning to forget
Li Jing, Caglar Gulcehre, John Peurifoy, Yichen Shen, Max Tegmark, Marin Soljacic, and Yoshua Bengio · 2019
Later among the works it cites.
Gated recurrent units viewed through the lens of continuous time dynamical systems
Ian D Jordan, Piotr Aleksander Sokol, and Il Memming Park · 2019
Later among the works it cites.
Non-normal recurrent neural network (nnrnn): learning long time dependencies while improving expressivity with transient dynamics
Giancarlo Kerg, Kyle Goyette, Maximilian Puelma Touzel, Gauthier Gidel, Eugene Vorontsov, Yoshua Bengio, and Guillaume Lajoie · 2019
Later among the works it cites.
Line attractor dynamics in recurrent networks for sentiment classification
Niru Maheswaranathan, Alex H Williams, Matthew D Golub, Surya Ganguli, and David Sussillo · 2019
Later among the works it cites.