A Rigorous Framework for the Mean Field Limit of Multilayer Neural Networks

Understand

We develop a mathematically rigorous framework for multilayer neural networks in the mean field regime.

• As the network's widths increase, the network's learning trajectory is shown to be well captured by a meaningful and dynamically nonlinear limit (the \textit{mean field} limit), which is characterized by a system of ODEs.
• Our framework applies to a broad range of network architectures, learning dynamics and network initializations.
• Central to the framework is the new idea of a \textit{neuronal embedding}, which comprises of a non-evolving probability space that allows to embed neural networks of arbitrary widths.