Normal approximation of Random Gaussian Neural Networks

Understand

In this paper we provide explicit upper bounds on some distances between the (law of the) output of a random Gaussian NN and (the law of) a random Gaussian vector.

• Our results concern both shallow random Gaussian neural networks with univariate output and fully connected and deep random Gaussian neural networks, with a rather general activation function.
• The upper bounds show how the widths of the layers, the activation functions and other architecture parameters affect the Gaussian approximation of the ouput.
• Our techniques, relying on Stein's method and integration by parts formulas for the Gaussian law, yield estimates on distances which are indeed integral probability metrics, and include the total variation and the convex distances.