Fetching the paper…
Reading the bibliography…
We provide a general framework for studying recurrent neural networks (RNNs) trained by injecting noise into hidden states.
Some properties of diffusion processes depending on a parameter
Yurii Nikolaevich Blagoveshchenskii and Mark Iosifovich Freidlin · 1961
Earlier work this paper cites.
Stochastic differential equations and stochastic flows of diffeomorphisms
Hiroshi Kunita · 1984
Earlier work this paper cites.
Lyapunov exponents of linear stochastic systems
Ludwig Arnold, W Kliemann, and E Oeljeklaus · 1986
Earlier work this paper cites.
Large deviations of linear stochastic differential equations
Ludwig Arnold and Wolfgang Kliemann · 1987
Earlier work this paper cites.
Exponential Stability of Stochastic Differential Equations
Xuerong Mao · 1994
Earlier work this paper cites.
Training with noise is equivalent to Tikhonov regularization
Chris M Bishop · 1995
Earlier work this paper cites.
An analysis of noise in recurrent neural networks: Convergence and generalization
Kam-Chuen Jim, C Lee Giles, and Bill G Horne · 1996
Earlier work this paper cites.
Noisy recurrent neural networks: the continuous-time case
S. Das and O. Olurotimi · 1998
Earlier work this paper cites.
Random Perturbations of Dynamical Systems
Mark Iosifovich Freidlin and Alexander D Wentzell · 1998
Earlier work this paper cites.
Brownian Motion
Ioannis Karatzas and Steven E Shreve · 1998
Earlier work this paper cites.
Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals
Ary L Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mietus, George B Moody, Chung-Kang Peng, and H Eugene Stanley · 2000
Earlier work this paper cites.
Strong convergence of Euler-type methods for nonlinear stochastic differential equations
Desmond J Higham, Xuerong Mao, and Andrew M Stuart · 2002
Earlier work this paper cites.
Nonlinear Systems
Hassan K Khalil and Jessy W Grizzle · 2002
Earlier work this paper cites.
Stochastic Differential Equations and Applications
Xuerong Mao · 2007
Earlier work this paper cites.
An introduction to SDE simulation
Simon JA Malham and Anke Wiese · 2010
Earlier work this paper cites.
Implementing regularization implicitly via approximate eigenvector computation
M. W. Mahoney and L. Orecchia · 2011
Earlier work this paper cites.
Strong convergence of an explicit numerical method for SDEs with nonglobally Lipschitz continuous coefficients
Martin Hutzenthaler, Arnulf Jentzen, Peter E Kloeden, et al · 2012
Earlier work this paper cites.
Approximate computation and implicit regularization for very large-scale data analysis
M. W. Mahoney · 2012
Earlier work this paper cites.
Robustness and generalization
Huan Xu and Shie Mannor · 2012
Earlier work this paper cites.
Numerical Solution of Stochastic Differential Equations
Peter E Kloeden and Eckhard Platen · 2013
Earlier work this paper cites.
On the difficulty of training recurrent neural networks
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio · 2013
Earlier work this paper cites.
Intriguing properties of neural networks
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus · 2013
Earlier work this paper cites.
Anti-differentiating approximation algorithms: A case study with min-cuts, spectral, and flow
D. F. Gleich and M. W. Mahoney · 2014
Earlier work this paper cites.
Understanding Machine Learning: From Theory to Algorithms
Shai Shalev-Shwartz and Shai Ben-David · 2014
Earlier work this paper cites.
Recurrent neural network regularization
Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals · 2014
Earlier work this paper cites.
Numerical approximations of stochastic differential equations with non-globally Lipschitz continuous coefficients
Martin Hutzenthaler and Arnulf Jentzen · 2015
Earlier work this paper cites.
A simple way to initialize recurrent networks of rectified linear units
Quoc V Le, Navdeep Jaitly, and Geoffrey E Hinton · 2015
Earlier work this paper cites.
Improving performance of recurrent neural network with ReLU nonlinearity
Sachin S Talathi and Aniket Vartak · 2015
Earlier work this paper cites.
Sequential neural models with stochastic layers
Marco Fraccaro, Søren Kaae Sø nderby, Ulrich Paquet, and Ole Winther · 2016
Earlier work this paper cites.
Brownian Motion, Martingales, and Stochastic Calculus
J.F.L. Gall · 2016
Earlier work this paper cites.
Deep networks with stochastic depth
Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q Weinberger · 2016
Cited alongside, same era.
On large-batch training for deep learning: Generalization gap and sharp minima
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang · 2016
Cited alongside, same era.
Stable architectures for deep neural networks
Eldad Haber and Lars Ruthotto · 2017
Cited alongside, same era.
Regularizing deep neural networks by noise: Its interpretation and optimization
Hyeonwoo Noh, Tackgeun You, Jonghwan Mun, and Bohyung Han · 2017
Cited alongside, same era.
Theory of deep learning III: Explaining the non-overfitting puzzle
Tomaso Poggio, Kenji Kawaguchi, Qianli Liao, Brando Miranda, Lorenzo Rosasco, Xavier Boix, Jack Hidary, and Hrushikesh Mhaskar · 2017
Cited alongside, same era.
Stability and convergence theory for learning ResNet: A full characterization
Huishuai Zhang, Da Yu, Mingyang Yi, Wei Chen, and Tie-yan Liu · 2019
Later among the works it cites.
Towards robust ResNet: A small step but a giant leap
Jingfeng Zhang, Bo Han, Laura Wynter, Bryan Kian Hsiang Low, and Mohan Kankanhalli · 2019
Later among the works it cites.
Symplectic ODE-Net: Learning Hamiltonian dynamics with control
Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty · 2019
Later among the works it cites.
The implicit regularization of stochastic gradient flow for least squares
Alnur Ali, Edgar Dobriban, and Ryan Tibshirani · 2020
Later among the works it cites.
Forecasting sequential data using consistent Koopman autoencoders
Omri Azencot, N Benjamin Erichson, Vanessa Lin, and Michael W. Mahoney · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Generalization error of deep neural networks: Role of classification margin and data structure
Jure Sokolić, Raja Giryes, Guillermo Sapiro, and Miguel RD Rodrigues · 2017
Cited alongside, same era.
Robust large margin deep neural networks
Jure Sokolić, Raja Giryes, Guillermo Sapiro, and Miguel RD Rodrigues · 2017
Cited alongside, same era.
Learning Koopman invariant subspaces for dynamic mode decomposition
Naoya Takeishi, Yoshinobu Kawahara, and Takehisa Yairi · 2017
Cited alongside, same era.
A proposal on machine learning via dynamical systems
E Weinan · 2017
Cited alongside, same era.
Dynamical isometry and a mean field theory of RNNs: Gating enables signal propagation in recurrent neural networks
Minmin Chen, Jeffrey Pennington, and Samuel Schoenholz · 2018
Cited alongside, same era.
Neural ordinary differential equations
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud · 2018
Cited alongside, same era.
Noisin: Unbiased regularization for recurrent neural networks
Adji Bousso Dieng, Rajesh Ranganath, Jaan Altosaar, and David Blei · 2018
Cited alongside, same era.
Kaushik Balakrishnan and Devesh Upadhyay · 2020
Later among the works it cites.
Explicit regularisation in Gaussian noise injections
Alexander Camuto, Matthew Willetts, Umut Simsekli, Stephen J Roberts, and Chris C Holmes · 2020
Later among the works it cites.
Exact expressions for double descent and implicit regularization via surrogate random design
Michal Derezinski, Feynman T Liang, and Michael W Mahoney · 2020
Later among the works it cites.
Optimizing neural networks via Koopman operator theory
Akshunna S. Dogra and William Redman · 2020
Later among the works it cites.
Lyapunov spectra of chaotic recurrent neural networks
Rainer Engelken, Fred Wolf, and LF Abbott · 2020
Later among the works it cites.
Adaptive Euler–Maruyama method for SDEs with nonglobally Lipschitz drift
Wei Fang, Michael B Giles, et al · 2020
Later among the works it cites.
RNNs incrementally evolving on an equilibrium manifold: A panacea for vanishing and exploding gradients?
Anil Kag, Ziming Zhang, and Venkatesh Saligrama · 2020
Later among the works it cites.
Neural controlled differential equations for irregular time series
Patrick Kidger, James Morrill, James Foster, and Terry Lyons · 2020
Later among the works it cites.
How does noise help robustness? Explanation and exploration under the neural SDE framework
Xuanqing Liu, Tesi Xiao, Si Si, Qin Cao, Sanjiv Kumar, and Cho-Jui Hsieh · 2020
Later among the works it cites.
Chao Ma, Stephan Wojtowytsch, and Lei Wu · 2020
Later among the works it cites.
Physics-informed probabilistic learning of linear embeddings of nonlinear dynamics with guaranteed stability
Shaowu Pan and Karthik Duraisamy · 2020
Later among the works it cites.
Continuous-in-depth neural networks
Alejandro F Queiruga, N Benjamin Erichson, Dane Taylor, and Michael W Mahoney · 2020
Later among the works it cites.
On Lyapunov exponents for RNNs: Understanding information propagation using dynamical systems tools
Ryan Vogt, Maximilian Puelma Touzel, Eli Shlizerman, and Guillaume Lajoie · 2020
Later among the works it cites.
The implicit and explicit regularization effects of dropout
Colin Wei, Sham Kakade, and Tengyu Ma · 2020
Later among the works it cites.
Dynamical system inspired adaptive time stepping controller for residual network families
Yibo Yang, Jianlong Wu, Hongyang Li, Xia Li, Tiancheng Shen, and Zhouchen Lin · 2020
Later among the works it cites.
Do RNN and LSTM have long memory?
Jingyu Zhao, Feiqing Huang, Jia Lv, Yanjie Duan, Zhen Qin, Guodong Li, and Guangjian Tian · 2020
Later among the works it cites.
Dropout: Explicit forms and capacity control
Raman Arora, Peter Bartlett, Poorya Mianjy, and Nathan Srebro · 2021
Closest in time.
Implicit bias of linear RNNs
Melikasadat Emami, Mojtaba Sahraee-Ardakan, Parthe Pandit, Sundeep Rangan, and Alyson K Fletcher · 2021
Closest in time.
Lipschitz recurrent neural networks
N. Benjamin Erichson, Omri Azencot, Alejandro Queiruga, Liam Hodgkinson, and Michael W. Mahoney · 2021
Closest in time.
Maxup: Lightweight adversarial training with data augmentation improves neural network training
Chengyue Gong, Tongzheng Ren, Mao Ye, and Qiang Liu · 2021
Closest in time.
Stochastic continuous normalizing flows: Training SDEs as ODEs
Liam Hodgkinson, Chris van der Heide, Fred Roosta, and Michael W Mahoney · 2021
Closest in time.
Understanding recurrent neural networks using nonequilibrium response theory
Soon Hoe Lim · 2021
Closest in time.
Coupled oscillatory recurrent neural network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies
T. Konstantin Rusch and Siddhartha Mishra · 2021
Closest in time.
On the origin of implicit regularization in stochastic gradient descent
Samuel L Smith, Benoit Dherin, David GT Barrett, and Soham De · 2021
Closest in time.
Understanding deep learning (still) requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals · 2021
Closest in time.