Fetching the paper…
Reading the bibliography…
In this paper, we study the infinite-depth limit of finite-width residual neural networks with random Gaussian weights.
“Theory of Financial Decision Making”
Jonathan. Ingersoll · 1987
Earlier work this paper cites.
“Bayesian Learning for Neural Networks”
R.M. Neal · 1995
Earlier work this paper cites.
“Numerical Solution of Stochastic Differential Equations”
Peter Kloeden and Eckhard Platen · 1995
Earlier work this paper cites.
“Stochastic Differential Equations”
Bernt Øksendal · 2003
Earlier work this paper cites.
“Nonlinear SDEs driven by Lévy processes and related PDEs”
Benjamin Jourdain, Sylvie Meleard and Wojbor Woyczynski · 2007
Earlier work this paper cites.
“Weak laws of large numbers for arrays under a condition of uniform integrability”
Soo Sung, Supranee Lisawadi and Andrei Volodin · 2008
Earlier work this paper cites.
“Exponential expressivity in deep neural networks through transient chaos”
B. Poole et al · 2016
Earlier work this paper cites.
“Deep Information Propagation”
S.S. Schoenholz, J. Gilmer, S. Ganguli and J. Sohl-Dickstein · 2017
Earlier work this paper cites.
“Mean field residual networks: On the edge of chaos”
G. Yang and S. Schoenholz · 2017
Earlier work this paper cites.
“Deep Neural Networks as Gaussian Processes”
J. Lee et al · 2018
Earlier work this paper cites.
“Gaussian Process Behaviour in Wide Deep Neural Networks”
A.G. Matthews et al · 2018
Earlier work this paper cites.
“CALCUL STOCHASTIQUE ET FINANCE”, 2018
Peter Tankov and Nizar Touzi · 2018
Earlier work this paper cites.
“On the Impact of the Activation Function on Deep Neural Networks Training”
S. Hayou, A. Doucet and J. Rousseau · 2019
Earlier work this paper cites.
“Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks”
Sanjeev Arora et al · 2019
Cited alongside, same era.
“ETraining dynamics of deep networks using stochastic gradient descent via neural tangent kernel”
Soufiane Hayou, Arnaud Doucet and Judith Rousseau · 2019
Cited alongside, same era.
“Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations”
Boris Hanin · 2019
Cited alongside, same era.
“Tensor Programs III: Neural Matrix Laws”
G. Yang · 2020
Cited alongside, same era.
“Infinite attention: NNGP and NTK for deep attention networks”
Jiri Hron, Yasaman Bahri, Jascha Sohl-Dickstein and Roman Novak · 2020
Cited alongside, same era.
“Regularization in ResNet with Stochastic Depth”
S. Hayou and F. Ayed · 2021
Later among the works it cites.
“Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping”
James Martens et al · 2021
Later among the works it cites.
“The future is log-Gaussian: ResNets and their infinite-depth-and-width limit at initialization”
Mufan Li, Mihai Nica and Dan Roy · 2021
Later among the works it cites.
“Connecting Optimization and Generalization via Gradient Flow Path Length”
Fusheng Liu, Haizhao Yang, Soufiane Hayou and Qianxiao Li · 2022
Closest in time.
“Analyzing Finite Neural Networks: Can We Trust Neural Tangent Kernel Theory?”
Mariia Seleznova and Gitta Kutyniok · 2022
Closest in time.
“The Curse of Depth in Kernel Regime”
Soufiane Hayou, Arnaud Doucet and Judith Rousseau · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
S. Hayou, A. Doucet and J. Rousseau · 2020
Cited alongside, same era.
“Disentangling Trainability and Generalization in Deep Neural Networks”
Lechao Xiao, Jeffrey Pennington and Samuel Schoenholz · 2020
Cited alongside, same era.
“Pruning untrained neural networks: Principles and analysis”
Soufiane Hayou, Jean-Francois Ton, Arnaud Doucet and Yee Teh · 2020
Cited alongside, same era.
“Bayesian Deep Ensembles via the Neural Tangent Kernel”
Bobby He, Balaji Lakshminarayanan and Yee Teh · 2020
Cited alongside, same era.
“Finite Depth and Width Corrections to the Neural Tangent Kernel”
Boris Hanin and Mihai Nica · 2020
Cited alongside, same era.
“Infinitely deep neural networks as diffusion processes”
Stefano Peluchetti and Stefano Favaro · 2020
Cited alongside, same era.
“Stable ResNet”
Soufiane Hayou et al · 2021
Cited alongside, same era.
Closest in time.
“Freeze and Chaos: NTK views on DNN Normalization, Checkerboard and Boundary Artifacts”
Arthur Jacot, Franck Gabriel, Francois Ged and Clement Hongler · 2022
Closest in time.
“Theory of Deep Learning: Neural Tangent Kernel and Beyond”
Arthur Jacot · 2022
Closest in time.
“Feature Learning and Signal Propagation in Deep Neural Networks”
Yizhang Lou, Chris Mingard and Soufiane Hayou · 2022
Closest in time.
“Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers”
Guodong Zhang, Aleksandar Botev and James Martens · 2022
Closest in time.
“The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization”
Mufan Li, Mihai Nica and Daniel. Roy · 2022
Closest in time.
“Correlation Functions in Random Fully Connected Neural Networks at Finite Width”
Boris Hanin · 2022
Closest in time.
“Scaling ResNets in the Large-depth Regime”
Pierre Marion, Adeline Fermanian, Gérard Biau and Jean-Philippe Vert · 2022
Closest in time.