Fetching the paper…
Reading the bibliography…
Viewing recurrent neural networks (RNNs) as continuous-time dynamical systems, we propose a recurrent unit that describes the hidden state's evolution with two parts: a well-understood linear component plus a Lipschitz nonlinearity.
Stability of motion , volume 138
Wolfgang Hahn · 1967
Earlier work this paper cites.
Learning distributed representations of concepts
Geoffrey E. Hinton · 1986
Earlier work this paper cites.
Dynamics and architecture for neural computation
Fernando J. Pineda · 1988
Earlier work this paper cites.
Approximation of dynamical systems by continuous time recurrent neural networks
Ken-ichi Funahashi and Yuichi Nakamura · 1993
Earlier work this paper cites.
TIMIT acoustic phonetic continuous speech corpus
John S. Garofolo · 1993
Earlier work this paper cites.
Building a large annotated corpus of English: the Penn Treebank
Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz · 1993
Earlier work this paper cites.
Learning long-term dependencies with gradient descent is difficult
Yoshua Bengio, Patrice Simard, and Paolo Frasconi · 1994
Earlier work this paper cites.
Gradient calculations for dynamic recurrent neural networks: A survey
Barak A. Pearlmutter · 1995
Earlier work this paper cites.
Nonlinear observer design using dynamic recurrent neural networks
Young H. Kim, Frank L. Lewis, and Chaouki T. Abdallah · 1996
Earlier work this paper cites.
Modeling of continuous time dynamical systems with input by recurrent neural networks
Tommy W. S. Chow and Xiao-Dong Li · 2000
Earlier work this paper cites.
Global stability of a class of continuous-time recurrent neural networks
Sanqing Hu and Jun Wang · 2002
Earlier work this paper cites.
Nonlinear Systems
Hassan K. Khalil · 2002
Earlier work this paper cites.
Approximation of dynamical time-variant systems by continuous-time recurrent neural networks
Xiao-Dong Li, John K. L. Ho, and Tommy W. S. Chow · 2005
Earlier work this paper cites.
Finite Difference Methods for Ordinary and Partial Differential Equations
Randall J. LeVeque · 2007
Earlier work this paper cites.
The tradeoffs of large scale learning
Léon Bottou and Olivier Bousquet · 2008
Earlier work this paper cites.
Matrix analysis
Roger A. Horn and Charles R. Johnson · 2012
Earlier work this paper cites.
Matrix analysis , volume 169
Rajendra Bhatia · 2013
Earlier work this paper cites.
On the difficulty of training recurrent neural networks
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio · 2013
Cited alongside, same era.
Nonlinear systems: Analysis, stability, and control , volume 10
Shankar Sastry · 2013
Cited alongside, same era.
Explaining and harnessing adversarial examples
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy · 2014
Cited alongside, same era.
A comprehensive review of stability analysis of continuous-time recurrent neural networks
Huaguang Zhang, Zhanshan Wang, and Derong Liu · 2014
Cited alongside, same era.
A simple way to initialize recurrent networks of rectified linear units
Quoc V. Le, Navdeep Jaitly, and Geoffrey E. Hinton · 2015
Cited alongside, same era.
Nais-net: Stable deep networks from non-autonomous differential equations
Marco Ciccone, Marco Gallieri, Jonathan Masci, Christian Osendorfer, and Faustino Gomez · 2018
Later among the works it cites.
Orthogonal recurrent neural networks with scaled Cayley transform
Kyle Helfrich, Devin Willmott, and Qiang Ye · 2018
Later among the works it cites.
Kronecker recurrent units
Cijo Jose, Moustapha Cisse, and Francois Fleuret · 2018
Later among the works it cites.
Fastgrnn: A fast, accurate, stable and tiny kilobyte sized gated recurrent neural network
Aditya Kusupati, Manish Singh, Kush Bhatia, Ashish Kumar, Prateek Jain, and Manik Varma · 2018
Later among the works it cites.
AntisymmetricRNN: A dynamical system view on recurrent neural networks
Bo Chang, Minmin Chen, Eldad Haber, and Ed Chi · 2019
Later among the works it cites.
Symplectic recurrent neural networks
Zhengdao Chen, Jianyu Zhang, Martin Arjovsky, and Léon Bottou · 2019
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Unitary evolution recurrent neural networks
Martin Arjovsky, Amar Shah, and Yoshua Bengio · 2016
Cited alongside, same era.
Recurrent orthogonal networks and long-memory tasks
Mikael Henaff, Arthur Szlam, and Yann LeCun · 2016
Cited alongside, same era.
Deepfool: A simple and accurate method to fool deep neural networks
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard · 2016
Cited alongside, same era.
Synthesis of recurrent neural networks for dynamical system simulation
Adam P. Trischler and Gabriele M. T. D’Eleuterio · 2016
Cited alongside, same era.
Full-capacity unitary recurrent neural networks
Scott Wisdom, Thomas Powers, John Hershey, Jonathan Le Roux, and Les Atlas · 2016
Cited alongside, same era.
Tunable efficient unitary neural networks (EUNN) and their application to RNNs
Li Jing, Yichen Shen, Tena Dubcek, John Peurifoy, Scott Skirlo, Yann LeCun, Max Tegmark, and Marin Soljačić · 2017
Cited alongside, same era.
The neural hawkes process: A neurally self-modulating multivariate point process
Hongyuan Mei and Jason M Eisner · 2017
Cited alongside, same era.
Later among the works it cites.
GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series
Edward De Brouwer, Jaak Simm, Adam Arany, and Yves Moreau · 2019
Later among the works it cites.
An investigation into neural net optimization via Hessian eigenvalue density
Behrooz Ghorbani, Shankar Krishnan, and Ying Xiao · 2019
Later among the works it cites.
Non-normal recurrent neural network (nnRNN): Learning long time dependencies while improving expressivity with transient dynamics
Giancarlo Kerg, Kyle Goyette, Maximilian Puelma Touzel, Gauthier Gidel, Eugene Vorontsov, Yoshua Bengio, and Guillaume Lajoie · 2019
Later among the works it cites.
Cheap orthogonal constraints in neural networks: A simple parametrization of the orthogonal and unitary group
Mario Lezcano-Casado and David Martinez-Rubio · 2019
Later among the works it cites.
Stable recurrent models
John Miller and Moritz Hardt · 2019
Later among the works it cites.
Recurrent neural networks in the eye of differential equations
Murphy Yuezhen Niu, Lior Horesh, and Isaac Chuang · 2019
Later among the works it cites.
Latent ordinary differential equations for irregularly-sampled time series
Yulia Rubanova, Tian Chen, and David Duvenaud · 2019
Later among the works it cites.
PyHessian: Neural networks through the lens of the Hessian
Zhewei Yao, Amir Gholami, Kurt Keutzer, and Michael W. Mahoney · 2019
Later among the works it cites.
RNNs incrementally evolving on an equilibrium manifold: A panacea for vanishing and exploding gradients?
Anil Kag, Ziming Zhang, and Venkatesh Saligrama · 2020
Closest in time.
Learning long-term dependencies in irregularly-sampled time series
Mathias Lechner and Ramin Hasani · 2020
Closest in time.
Momentumrnn: Integrating momentum into recurrent neural networks
Tan M. Nguyen, Richard G. Baraniuk, Andrea L. Bertozzi, Stanley J. Osher, and Bao Wang · 2020
Closest in time.