Fetching the paper…
Reading the bibliography…
The design of recurrent neural networks (RNNs) to accurately process sequential inputs with long-time dependencies is very challenging on account of the exploding and vanishing gradient problem.
Mathematical methods of classical mechanics
Arnold, V. I · 1989
Earlier work this paper cites.
Nonlinear oscillations, dynamical systems, and bifurcations of vector fields
Guckenheimer, J. and Holmes, P · 1990
Earlier work this paper cites.
Backpropagation through time: what it does and how to do it
Werbos, P. J · 1990
Earlier work this paper cites.
Numerical Hamiltonian problems
Sanz Serna, J. and Calvo, M · 1994
Earlier work this paper cites.
Predictability: A problem partly solved
Lorenz, E. N · 1996
Earlier work this paper cites.
Long short-term memory
Hochreiter, S. and Schmidhuber, J · 1997
Earlier work this paper cites.
Gradient-based learning applied to document recognition
LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P · 1998
Earlier work this paper cites.
Geometric numerical integration illustrated by the störmer-verlet method
Hairer, E., Lubich, C., and Wanner, G · 2003
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G., et al · 2009
Earlier work this paper cites.
Learning word vectors for sentiment analysis
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., and Potts, C · 2011
Earlier work this paper cites.
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., and Bengio, Y · 2013
Earlier work this paper cites.
Learning phrase representations using rnn encoder-decoder for statistical machine translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bougares, F., Schwenk, H., and Bengio, Y · 2014
Earlier work this paper cites.
Glove: Global vectors for word representation
Pennington, J., Socher, R., and Manning, C. D · 2014
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
He, K., Zhang, X., Ren, S., and Sun, J · 2015
Earlier work this paper cites.
A simple way to initialize recurrent networks of rectified linear units
Le, Q. V., Jaitly, N., and Hinton, G. E · 2015
Earlier work this paper cites.
Deep learning
LeCun, Y., Bengio, Y., and Hinton, G · 2015
Earlier work this paper cites.
Nonlinear Dynamics and Chaos
Strogatz, S · 2015
Cited alongside, same era.
Optimizing performance of recurrent neural networks on gpus
Appleyard, J., Kocisky, T., and Blunsom, P · 2016
Cited alongside, same era.
Unitary evolution recurrent neural networks
Arjovsky, M., Shah, A., and Bengio, Y · 2016
Cited alongside, same era.
A theoretically grounded application of dropout in recurrent neural networks
Gal, Y. and Ghahramani, Z · 2016
Cited alongside, same era.
Recurrent orthogonal networks and long-memory tasks
Henaff, M., Szlam, A., and LeCun, Y · 2016
Cited alongside, same era.
Toward a robust estimation of respiratory rate from pulse oximeters
Pimentel, M. A., Johnson, A. E., Charlton, P. H., Birrenkott, D., Watkinson, P. J., Tarassenko, L., and Clifton, D. A · 2016
Simple recurrent units for highly parallelizable recurrence
Lei, T., Zhang, Y., Wang, S. I., Dai, H., and Artzi, Y · 2018
Later among the works it cites.
Independently recurrent neural network (indrnn): Building a longer and deeper rnn
Li, S., Li, W., Cook, C., Zhu, C., and Gao, Y · 2018
Later among the works it cites.
Reversible recurrent neural networks
MacKay, M., Vicol, P., Ba, J., and Grosse, R. B · 2018
Later among the works it cites.
Trivializations for gradient-based optimization on manifolds
Casado, M. L · 2019
Later among the works it cites.
Cheap orthogonal constraints in neural networks: A simple parametrization of the orthogonal and unitary group
Casado, M. L. and Martínez-Rubio, D · 2019
Later among the works it cites.
Hamiltonian neural networks
Greydanus, S., Dzamba, M., and Yosinski, J · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Dilated recurrent neural networks
Chang, S., Zhang, Y., Han, W., Yu, M., Guo, X., Tan, W., Cui, X., Witbrock, M., Hasegawa-Johnson, M. A., and Huang, T. S · 2017
Cited alongside, same era.
Gate-variants of gated recurrent unit (gru) neural networks
Dey, R. and Salemt, F. M · 2017
Cited alongside, same era.
A proposal on machine learning via dynamical systems
E, W · 2017
Cited alongside, same era.
A recurrent neural network without chaos
Laurent, T. and von Brecht, J · 2017
Cited alongside, same era.
The uea multivariate time series classification archive, 2018
Bagnall, A., Dau, H. A., Lines, J., Flynn, M., Large, J., Bostrom, A., Southam, P., and Keogh, E · 2018
Cited alongside, same era.
Skip RNN: learning to skip state updates in recurrent neural networks
Campos, V., Jou, B., Giró-i-Nieto, X., Torres, J., and Chang, S · 2018
Cited alongside, same era.
Non-normal recurrent neural network (nnrnn): learning long time dependencies while improving expressivity with transient dynamics
Kerg, G., Goyette, K., Touzel, M. P., Gidel, G., Vorontsov, E., Bengio, Y., and Lajoie, G · 2019
Later among the works it cites.
Deep independently recurrent neural network (indrnn)
Li, S., Li, W., Cook, C., Gao, Y., and Zhu, C · 2019
Later among the works it cites.
Normalizing flows for probabilistic modeling and inference
Papamakarios, G., Nalisnick, E., Rezende, D. J., Mohamed, S., and Lakshminarayanan, B · 2019
Later among the works it cites.
Symplectic recurrent neural networks
Chen, Z., Zhang, J., Arjovsky, M., and Bottou, L · 2020
Later among the works it cites.
Rnns incrementally evolving on an equilibrium manifold: A panacea for vanishing and exploding gradients?
Kag, A., Zhang, Z., and Saligrama, V · 2020
Later among the works it cites.
Neural cdes for long time series via the log-ode method
Morrill, J., Kidger, P., Salvi, C., Foster, J., and Lyons, T · 2020
Later among the works it cites.
Monash university, uea, ucr time series regression archive
Tan, C. W., Bergmeir, C., Petitjean, F., and Webb, G. I · 2020
Later among the works it cites.
Lipschitz recurrent neural networks
Erichson, N. B., Azencot, O., Queiruga, A., and Mahoney, M. W · 2021
Closest in time.
Coupled oscillatory recurrent neural network (cornn): An accurate and (gradient) stable architecture for learning long time dependencies
Rusch, T. K. and Mishra, S · 2021
Closest in time.
Full-capacity unitary recurrent neural networks
Wisdom, S., Powers, T., Hershey, J., Le Roux, J., and Atlas, L · 2080
Closest in time.