Fetching the paper…
Reading the bibliography…
Recent work has attempted to interpret residual networks (ResNets) as one step of a forward Euler discretization of an ordinary differential equation, focusing mainly on syntactic algebraic similarities between the two systems.
Neural SDE: Stabilizing neural ode networks with stochastic noise
Xuanqing Liu, Tesi Xiao, Si Si, Qin Cao, Sanjiv Kumar, and Cho-Jui Hsieh · 1906
Earlier work this paper cites.
Approximation capabilities of neural ordinary differential equations
Han Zhang, Xi Gao, Jacob Unterman, and Tom Arodz · 1907
Earlier work this paper cites.
An iterative method of solving elliptic net problems
G.P. Astrakhantsev · 1971
Earlier work this paper cites.
Differential equations, dynamical systems, and linear algebra , volume 60
Morris W Hirsch, Robert L Devaney, and Stephen Smale · 1974
Earlier work this paper cites.
Learning to predict by the methods of temporal differences
Richard S Sutton · 1988
Earlier work this paper cites.
Learning to control an unstable system with forward modeling
Michael I Jordan and Robert A Jacobs · 1990
Earlier work this paper cites.
Gradient and hamiltonian dynamics applied to learning in neural networks
James W Howse, Chaouki T Abdallah, and Gregory L Heileman · 1996
Earlier work this paper cites.
Stefano Massaroli, Michael Poli, Jinkyoo Park, Atsushi Yamashita, and Hajime Asama · 2002
Earlier work this paper cites.
Stefano Massaroli, Michael Poli, Michelangelo Bin, Jinkyoo Park, Atsushi Yamashita, and Hajime Asama · 2003
Earlier work this paper cites.
Exact solution for the nonlinear pendulum
Augusto Beléndez, Carolina Pascual, DI Méndez, Tarsicio Beléndez, and Cristian Neipp · 2007
Earlier work this paper cites.
Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems , volume 98
Randall J LeVeque · 2007
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky et al · 2009
Earlier work this paper cites.
On the importance of initialization and momentum in deep learning
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Earlier work this paper cites.
Identity matters in deep learning
Moritz Hardt and Tengyu Ma · 2016
Earlier work this paper cites.
Fractalnet: Ultra-deep neural networks without residuals
Gustav Larsson, Michael Maire, and Gregory Shakhnarovich · 2016
Earlier work this paper cites.
A differential equation for modeling Nesterov’s accelerated gradient method: Theory and insights
Weijie Su, Stephen Boyd, and Emmanuel J Candes · 2016
Earlier work this paper cites.
Residual networks behave like ensembles of relatively shallow networks
Andreas Veit, Michael J Wilber, and Serge Belongie · 2016
Earlier work this paper cites.
Sergey Zagoruyko and Nikos Komodakis · 2016
Earlier work this paper cites.
The shattered gradients problem: If ResNets are the answer, then what is the question?
David Balduzzi, Marcus Frean, Lennox Leary, JP Lewis, Kurt Wan-Duo Ma, and Brian McWilliams · 2017
Earlier work this paper cites.
A survey of model compression and acceleration for deep neural networks
Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang · 2017
Earlier work this paper cites.
The reversible residual network: Backpropagation without storing activations
Aidan N Gomez, Mengye Ren, Raquel Urtasun, and Roger B Grosse · 2017
Earlier work this paper cites.
Stable architectures for deep neural networks
Eldad Haber and Lars Ruthotto · 2017
Earlier work this paper cites.
Densely connected convolutional networks
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger · 2017
Earlier work this paper cites.
Maximum principle based algorithms for deep learning
Qianxiao Li, Long Chen, Cheng Tai, and E Weinan · 2017
Earlier work this paper cites.
Convergence analysis of two-layer neural networks with ReLU activation
Yuanzhi Li and Yang Yuan · 2017
Earlier work this paper cites.
Automatic differentiation in Pytorch
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer · 2017
Cited alongside, same era.
A proposal on machine learning via dynamical systems
E Weinan · 2017
Cited alongside, same era.
Tiny ImageNet challenge
Jiayu Wu, Qixiang Zhang, and Guoxi Xu · 2017
Cited alongside, same era.
Aggregated residual transformations for deep neural networks
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He · 2017
Cited alongside, same era.
Mean field residual networks: On the edge of chaos
Greg Yang and Samuel Schoenholz · 2017
Cited alongside, same era.
Polynet: A pursuit of structural diversity in very deep networks
Xingcheng Zhang, Zhizhong Li, Chen Change Loy, and Dahua Lin · 2017
Cited alongside, same era.
Equivariant flows: Sampling configurations for multi-body systems with symmetric energies
Jonas Köhler, Leon Klein, and Frank Noé · 2019
Later among the works it cites.
Deep learning theory review: An optimal control and dynamical systems perspective
Guan-Horng Liu and Evangelos A Theodorou · 2019
Later among the works it cites.
Lu Lu, Pengzhan Jin, and George Em Karniadakis · 2019
Later among the works it cites.
A dynamical systems perspective on nesterov acceleration
Michael Muehlebach and Michael Jordan · 2019
Later among the works it cites.
Shadowing properties of optimization algorithms
Antonio Orvieto and Aurelien Lucchi · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Neural ordinary differential equations
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud · 2018
Cited alongside, same era.
NAIS-Net: Stable deep networks from non-autonomous differential equations
Marco Ciccone, Marco Gallieri, Jonathan Masci, Christian Osendorfer, and Faustino Gomez · 2018
Cited alongside, same era.
FFJORD: Free-form continuous dynamics for scalable reversible generative models
Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud · 2018
Cited alongside, same era.
Orthogonal recurrent neural networks with scaled Cayley transform
Kyle Helfrich, Devin Willmott, and Qiang Ye · 2018
Cited alongside, same era.
Visualizing the loss landscape of neural nets
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein · 2018
Cited alongside, same era.
An optimal control approach to deep learning and applications to discrete-weight neural networks
Qianxiao Li and Shuji Hao · 2018
Cited alongside, same era.
Graph neural ordinary differential equations
Michael Poli, Stefano Massaroli, Junyoung Park, Atsushi Yamashita, Hajime Asama, and Jinkyoo Park · 2019
Later among the works it cites.
SNODE: Spectral discretization of neural ODEs for system identification
Alessio Quaglino, Marco Gallieri, Jonathan Masci, and Jan Koutník · 2019
Later among the works it cites.
Residual networks as nonlinear systems: Stability analysis using linearization
Kai Rothauge, Zhewei Yao, Zixi Hu, and Michael W Mahoney · 2019
Later among the works it cites.
Latent ordinary differential equations for irregularly-sampled time series
Yulia Rubanova, Ricky T. Q. Chen, and David K Duvenaud · 2019
Later among the works it cites.
Deep neural networks motivated by partial differential equations
Lars Ruthotto and Eldad Haber · 2019
Later among the works it cites.
Hamiltonian graph networks with ode integrators
Alvaro Sanchez-Gonzalez, Victor Bapst, Kyle Cranmer, and Peter Battaglia · 2019
Later among the works it cites.
Transport analysis of infinitely deep neural network
Sho Sonoda and Noboru Murata · 2019
Later among the works it cites.
HAQ: Hardware-aware automated quantization with mixed precision
Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han · 2019
Later among the works it cites.
Pointflow: 3d point cloud generation with continuous normalizing flows
Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan · 2019
Later among the works it cites.
Forward stability of resnet and its variants
Linan Zhang and Hayden Schaeffer · 2019
Later among the works it cites.
Symplectic ODE-Net: Learning Hamiltonian dynamics with control
Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty · 2019
Later among the works it cites.
Forecasting sequential data using consistent Koopman autoencoders
Omri Azencot, N Benjamin Erichson, Vanessa Lin, and Michael W Mahoney · 2020
Closest in time.
Lagrangian neural networks
Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, and Shirley Ho · 2020
Closest in time.
Batch normalization biases deep residual networks towards shallow paths
Soham De and Samuel L Smith · 2020
Closest in time.
Lipschitz recurrent neural networks
N Benjamin Erichson, Omri Azencot, Alejandro Queiruga, and Michael W Mahoney · 2020
Closest in time.
Chris Finlay, Jörn-Henrik Jacobsen, Levon Nurbekyan, and Adam M Oberman · 2020
Closest in time.
Neural ordinary differential equation based recurrent neural network model
Mansura Habiba and Barak A Pearlmutter · 2020
Closest in time.
Scalable gradients for stochastic differential equations
Xuechen Li, Ting-Kam Leonard Wong, Ricky TQ Chen, and David Duvenaud · 2020
Closest in time.
Understanding recurrent neural networks using nonequilibrium response theory
Soon Hoe Lim · 2020
Closest in time.
Universal differential equations for scientific machine learning
Christopher Rackauckas, Yingbo Ma, Julius Martensen, Collin Warner, Kirill Zubov, Rohit Supekar, Dominic Skinner, and Ali Ramadhan · 2020
Closest in time.
And the bit goes down: Revisiting the quantization of neural networks
Pierre Stock, Armand Joulin, Rémi Gribonval, Benjamin Graham, and Hervé Jégou · 2020
Closest in time.
Adaptive checkpoint adjoint method for gradient estimation in neural ODE
Juntang Zhuang, Nicha Dvornek, Xiaoxiao Li, Sekhar Tatikonda, Xenophon Papademetris, and James Duncan · 2020
Closest in time.