Exact posterior distributions of wide Bayesian neural networks

Scaling limits of wide neural networks with weight sharing: Gaussian process behavior, gradient independence, and neural tangent kernel derivation

Original

Yang, G · 1902

Earlier work this paper cites.

Central limit theorems for interchangeable processes

Blum, J. R., Chernoff, H., Rosenblatt, M., and Teicher, H · 1958

Earlier work this paper cites.

Probability and Measure

Billingsley, P · 1986

Earlier work this paper cites.

Bayesian Learning for Neural Networks

Neal, R. M · 1996

Earlier work this paper cites.

Gaussian processes for machine learning , volume 1

Rasmussen, C. E. and Williams, C. K · 2006

Earlier work this paper cites.

Theory of statistics

Schervish, M. J · 2012

Earlier work this paper cites.

Neural tangent kernel: Convergence and generalization in neural networks

Jacot, A., Gabriel, F., and Hongler, C · 2018

Cited alongside, same era.

Deep neural networks as Gaussian processes

Lee, J., Sohl-dickstein, J., Pennington, J., Novak, R., Schoenholz, S., and Bahri, Y · 2018

Cited alongside, same era.

Gaussian process behaviour in wide deep neural networks

Original

Matthews, A. G., Rowland, M., Hron, J., Turner, R. E., and Ghahramani, Z · 2018

Cited alongside, same era.

Deep convolutional networks as shallow Gaussian processes

Garriga-Alonso, A., Rasmussen, C. E., and Aitchison, L · 2019

Cited alongside, same era.

Wide neural networks of any depth evolve as linear models under gradient descent

Lee, J., Xiao, L., Schoenholz, S. S., Bahri, Y., Novak, R., Sohl-Dickstein, J., and Pennington, J · 2019

Cited alongside, same era.

Bayesian deep convolutional networks with many channels are Gaussian processes

Novak, R., Xiao, L., Bahri, Y., Lee, J., Yang, G., Hron, J., Abolafia, D. A., Pennington, J., and Sohl-Dickstein, J · 2019

Later among the works it cites.

Why bigger is not always better: on finite and infinite neural networks

Aitchison, L · 2020

Closest in time.

Infinite attention: NNGP and NTK for deep attention networks

Hron, J., Bahri, Y., Sohl-Dickstein, J., and Novak, R · 2020

Closest in time.

Neural tangents: Fast and easy infinite neural networks in python

Novak, R., Xiao, L., Hron, J., Lee, J., Alemi, A. A., Sohl-Dickstein, J., and Schoenholz, S. S · 2020

Closest in time.

On the infinite width limit of neural networks with a standard parameterization

Original

Sohl-Dickstein, J., Novak, R., Schoenholz, S. S., and Lee, J · 2020

Closest in time.