Fetching the paper…

Reading the bibliography…

2022

Asymptotic Analysis of Deep Residual Networks

Cont, Rama, Rossier, Alain, Xu, Renyuan

Original
PDF
Post on X

Understand

We investigate the asymptotic properties of deep Residual networks (ResNets) as the number of layers increases.

We first show the existence of scaling regimes for trained weights markedly different from those implicitly assumed in the neural ODE literature.
We study the convergence of the hidden state dynamics in these scaling regimes, showing that one may obtain an ODE, a stochastic differential equation (SDE) or neither of these.
In particular, our findings point to the existence of a diffusive regime in which the deep network limit is described by a class of stochastic differential equations (SDEs).

Asymptotic Analysis of Deep Residual Networks · Around

Built on

M. Emery
1979
Earlier work this paper cites.
H. M. Dietz
1991
Earlier work this paper cites.
Y. Bengio, P. Simard, and P. Frasconi
1994
Earlier work this paper cites.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner
1998
Earlier work this paper cites.
D. J. Higham, X. Mao, and A. M. Stuart
2002
Earlier work this paper cites.
A. Krizhevsky, G. Hinton, et al
2009
Earlier work this paper cites.
E. Platen and N. Bruti-Liberati
2010
Earlier work this paper cites.
D. Revuz and M. Yor
2013
Earlier work this paper cites.
S. Ioffe and C. Szegedy
Original
2015
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun
2016
Earlier work this paper cites.
J. Heymann, L. Drude, and R. Haeb-Umbach
2016
Earlier work this paper cites.

Similar

G. Huang, Y. Sun, Z. Liu, D. Sedra, and K. Weinberger
Original
2016
Cited alongside, same era.
I. Loshchilov and F. Hutter
Original
2016
Cited alongside, same era.
S. Mallat
2016
Cited alongside, same era.
S. Zagoruyko and N. Komodakis
Original
2016
Cited alongside, same era.
R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud
2018
Cited alongside, same era.
E. Haber and L. Ruthotto
Original

Then

W. E, C. Ma, Q. Wang, and L. Wu
Original
2019
Later among the works it cites.
S. Frei, Y. Cao, and Q. Gu
Original
2019
Later among the works it cites.
X. Li, T.-K. L. Wong, R. T. Q. Chen, and D. K. Duvenaud
2020
Later among the works it cites.
Y. Lu, C. Ma, Y. Lu, J. Lu, and L. Ying
2020
Later among the works it cites.
S. Peluchetti and S. Favaro
2020
Later among the works it cites.
T. Bachlechner, B. P. Majumder, H. Mao, G. Cottrell, and J. McAuley
2021
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2018

Cited alongside, same era.

H. Sedghi, V. Gupta, and P. M. Long

2018

Cited alongside, same era.

S. Arora, N. Cohen, N. Golowich, and W. Hu

2019

Cited alongside, same era.

E. Dupont, A. Doucet, and Y. W. Teh

2019

Cited alongside, same era.

W. E, J. Han, and Q. Li

2019

Cited alongside, same era.

K. Ott, P. Katiyar, P. Hennig, and M. Tiemann

2021

Later among the works it cites.

M. E. Sander, P. Ablin, and G. Peyré

2022

Closest in time.

M. Thorpe and Y. van Gennip

2023

Closest in time.

A.-S. Cohen, R. Cont, A. Rossier, and R. Xu

2048

Closest in time.