Non-Gaussian processes and neural networks at finite widths

Understand

Gaussian processes are ubiquitous in nature and engineering.

• A case in point is a class of neural networks in the infinite-width limit, whose priors correspond to Gaussian processes.
• Here we perturbatively extend this correspondence to finite-width neural networks, yielding non-Gaussian processes as priors.
• The methodology developed herein allows us to track the flow of preactivation distributions by progressively integrating out random variables from lower to higher layers, reminiscent of renormalization-group flow.