Scaling Limit of Neural Networks with the Xavier Initialization and Convergence to a Global Minimum

Markov Processes: Characterization and Convergence

S. Ethier and T. Kurtz · 1986

Earlier work this paper cites.

Multilayer feedforward networks are universal approximators

K. Hornik, M. Stinchcombe, and H. White · 1989

Earlier work this paper cites.

Approximation capabilities of multilayer feedforward networks

K. Hornik · 1991

Earlier work this paper cites.

Convergence of learning algorithms with constant learning rates

C. Kuan and K. Hornik · 1991

Earlier work this paper cites.

Nonlinearity creates linear independence

Yoshifusa Ito · 1996

Earlier work this paper cites.

Understanding the difficulty of training deep feedforward neural networks

X. Glorot and Y. Bengio · 2010

Cited alongside, same era.

Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks

D. Zou, Y. Cao, D. Zhou, and Q. Gu · 2018

Cited alongside, same era.

Neural Tangent Kernel: Convergence and Generalization in Neural Networks

A. Jacot, F. Gabriel, and C. Hongler · 2018

Cited alongside, same era.

A mean field view of the landscape of two-layer neural networks

S. Mei, A. Montanari, and P. Nguyen · 2018

Cited alongside, same era.

Neural Networks as Interacting Particle Systems: Asymptotic Convexity of the Loss Landscape and Universal Scaling of the Approximation Error

Original

G. M. Rotskoff and E. Vanden-Eijnden · 2018

Cited alongside, same era.

Gradient Descent Finds Global Minima of Deep Neural Networks

S. Du, J. Lee, H. Li, L. Wang, and X. Zhai · 2019

Closest in time.

Gradient Descent Provably Optimizes Over-Parameterized Neural Networks

S. Du, X. Zhai, B. Poczos, and A. Singh · 2019

Closest in time.

Mean Field Analysis of Neural Networks: A Law of Large Numbers

J. Sirignano and K. Spiliopoulos · 2020

Closest in time.

Mean Field Analysis of Neural Networks: A Central Limit Theorem

J. Sirignano and K. Spiliopoulos · 2020

Closest in time.

Mean Field Analysis of Deep Neural Networks

J. Sirignano and K. Spiliopoulos · 2021

Closest in time.