Fetching the paper…
Reading the bibliography…
Recurrent neural networks (RNNs) have been successfully applied to a variety of problems involving sequential data, but their optimization is sensitive to parameter initialization, architecture, and optimizer hyperparameters.
Lyapunov characteristic exponents for smooth dynamical systems and for hamiltonian systems; a method for computing all of them. part 1: Theory
Giancarlo Benettin, Luigi Galgani, Antonio Giorgilli, and Jean-Marie Strelcyn · 1980
Earlier work this paper cites.
Random Dynamical Systems
Ludwig Arnold · 1991
Earlier work this paper cites.
Suppressing chaos in neural networks by noise
L. Molgedey, J. Schuchhardt, and H. G. Schuster · 1992
Earlier work this paper cites.
Learning long-term dependencies with gradient descent is difficult
Yoshua Bengio, Patrice Simard, and Paolo Frasconi · 1994
Earlier work this paper cites.
Computation of a few Lyapunov exponents for continuous and discrete dynamical systems
L Dieci and E S Van Vleck · 1995
Earlier work this paper cites.
Edge of chaos and prediction of computational performance for neural circuit models
R Legenstein and W Maass · 2007
Earlier work this paper cites.
Dynamical entropy production in spiking neuron networks in the balanced state
Michael Monteforte and Fred Wolf · 2010
Earlier work this paper cites.
Non-normal amplification in random balanced neuronal networks
Guillaume Hennequin, Tim Vogels, and Wulfram Gerstner · 2012
Earlier work this paper cites.
On the difficulty of training recurrent neural networks
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio · 2013
Earlier work this paper cites.
Chaos and reliability in balanced spiking networks with temporal drive
Guillaume Lajoie, Kevin K. Lin, and Eric Shea-Brown · 2013
Earlier work this paper cites.
Cellular dynamics and stable chaos in balanced networks
Maximilian Puelma Touzel · 2015
Cited alongside, same era.
Visualizing and understanding recurrent networks
Andrej Karpathy, Justin Johnson, and Li Fei-Fei · 2015
Cited alongside, same era.
Exponential expressivity in deep neural networks through transient chaos
Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, and Surya Ganguli · 2016
Cited alongside, same era.
A recurrent neural network without chaos
Thomas Laurent and James von Brecht · 2016
Cited alongside, same era.
Recurrent orthogonal networks and long-memory tasks
Mikael Henaff, Arthur Szlam, and Yann LeCun · 2016
Cited alongside, same era.
Optimal Sequence Memory in Driven Random Networks
Jannis Schuecker, Sven Goedeke, and Moritz Helias · 2018
Later among the works it cites.
full-force: A target-based method for training recurrent networks
Brian DePasquale, Christopher J Cueva, Kanaka Rajan, G Sean Escola, and LF Abbott · 2018
Later among the works it cites.
Greg Yang · 2019
Later among the works it cites.
Dynamical isometry and a mean field theory of lstms and grus
Dar Gilboa, Bo Chang, Minmin Chen, Greg Yang, Samuel S Schoenholz, Ed H Chi, and Jeffrey Pennington · 2019
Later among the works it cites.
Non-normal recurrent neural network (nnrnn): learning long time dependencies while improving expressivity with transient dynamics
Giancarlo Kerg, Kyle Goyette, Maximilian Puelma Touzel, Gauthier Gidel, Eugene Vorontsov, Yoshua Bengio, and Guillaume Lajoie · 2019
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
Jeffrey Pennington, Samuel Schoenholz, and Surya Ganguli · 2017
Cited alongside, same era.
Gating Enables Signal Propagation in Recurrent Neural Networks
MinMin Chen, Jeffery Pennington, and Samuel · 2018
Cited alongside, same era.
The emergence of spectral universality in deep networks
Jeffrey Pennington, Samuel S Schoenholz, and Surya Ganguli · 2018
Cited alongside, same era.
Convolutional sequence to sequence model for human dynamics
Chen Li, Zhen Zhang, Wee Sun Lee, and Gim Hee Lee · 2018
Cited alongside, same era.
Later among the works it cites.
Deep learning theory review: An optimal control and dynamical systems perspective
Guan-Horng Liu and Evangelos A Theodorou · 2019
Later among the works it cites.
R-force: Robust learning for random recurrent neural networks
Yang Zheng and Eli Shlizerman · 2020
Closest in time.
Lyapunov spectra of chaotic recurrent neural networks
Rainer Engelken, Fred Wolf, and LF Abbott · 2020
Closest in time.
Gating creates slow modes and controls phase-space complexity in grus and lstms
Tankut Can, Kamesh Krishnamurthy, and David J Schwab · 2020
Closest in time.