Fetching the paper…
Reading the bibliography…
Wide neural networks with random weights and biases are Gaussian processes, as originally observed by Neal (1995) and more recently by Lee et al.
A Mean Field Theory of Batch Normalization
Greg Yang, Jeffrey Pennington, Vinay Rao, Jascha Sohl-Dickstein, and Samuel S. Schoenholz · 1902
Earlier work this paper cites.
Echo state network
Herbert Jaeger · 1941
Earlier work this paper cites.
Cognitron: A self-organizing multilayered neural network
Kunihiko Fukushima · 1975
Earlier work this paper cites.
Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition
Kunihiko Fukushima and Sei Miyake · 1982
Earlier work this paper cites.
Learning internal representations by error propagation
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams · 1985
Earlier work this paper cites.
BAYESIAN LEARNING FOR NEURAL NETWORKS
Radford M Neal · 1995
Earlier work this paper cites.
Long Short-Term Memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Computing with Infinite Networks
Christopher K I Williams · 1997
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
Object recognition with gradient-based learning
Yann LeCun, Patrick Haffner, Léon Bottou, and Yoshua Bengio · 1999
Earlier work this paper cites.
Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
Wolfgang Maass, Thomas Natschläger, and Henry Markram · 2002
Earlier work this paper cites.
Hierarchical Gaussian process latent variable models
Neil D Lawrence and Andrew J Moore · 2007
Earlier work this paper cites.
Continuous neural networks
Nicolas Le Roux and Yoshua Bengio · 2007
Earlier work this paper cites.
An overview of reservoir computing: Theory, applications and implementations
Benjamin Schrauwen · 2007
Earlier work this paper cites.
Kernel methods for deep learning
Youngmin Cho and Lawrence K. Saul · 2009
Earlier work this paper cites.
The dynamics of message passing on dense graphs, with applications to compressed sensing
Mohsen Bayati and Andrea Montanari · 2010
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio · 2010
Earlier work this paper cites.
Functional analysis: introduction to further topics in analysis
Elias M. Stein and Rami Shakarchi · 2011
Earlier work this paper cites.
An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model
Erwin Bolthausen · 2012
Earlier work this paper cites.
Spectral Networks and Locally Connected Networks on Graphs
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun · 2013
Earlier work this paper cites.
Deep gaussian processes
Andreas Damianou and Neil Lawrence · 2013
Earlier work this paper cites.
Neural Machine Translation by Jointly Learning to Align and Translate
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio · 2014
Cited alongside, same era.
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio · 2014
Cited alongside, same era.
Glove: Global Vectors for Word Representation
Jeffrey Pennington, Richard Socher, and Christopher Manning · 2014
Cited alongside, same era.
Convolutional Networks on Graphs for Learning Molecular Fingerprints
David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alan Aspuru-Guzik, and Ryan P Adams · 2015
Cited alongside, same era.
Steps Toward Deep Kernel Methods from Infinite Neural Networks
Tamir Hazan and Tommi Jaakkola · 2015
Deep Information Propagation
Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, and Jascha Sohl-Dickstein · 2017
Later among the works it cites.
Convolutional Gaussian Processes
Mark van der Wilk, Carl Edward Rasmussen, and James Hensman · 2017
Later among the works it cites.
Attention is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \Lukasz Kaiser, and Illia Polosukhin · 2017
Later among the works it cites.
Mean Field Residual Network: On the Edge of Chaos
Greg Yang and Samuel S. Schoenholz · 2017
Later among the works it cites.
Deep convolutional Gaussian processes
Kenneth Blomqvist, Samuel Kaski, and Markus Heinonen · 2018
Later among the works it cites.
A gaussian process perspective on convolutional neural networks
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Cited alongside, same era.
Deep Convolutional Networks on Graph-Structured Data
Mikael Henaff, Joan Bruna, and Yann LeCun · 2015
Cited alongside, same era.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe and Christian Szegedy · 2015
Cited alongside, same era.
Gated Graph Sequence Neural Networks
Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel · 2015
Cited alongside, same era.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton · 2016
Cited alongside, same era.
Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity
Amit Daniely, Roy Frostig, and Yoram Singer · 2016
Cited alongside, same era.
Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst · 2016
Cited alongside, same era.
Anastasia Borovykh · 2018
Later among the works it cites.
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks
Minmin Chen, Jeffrey Pennington, and Samuel Schoenholz · 2018
Later among the works it cites.
Deep Convolutional Networks as shallow Gaussian Processes
Adrià Garriga-Alonso, Laurence Aitchison, and Carl Edward Rasmussen · 2018
Later among the works it cites.
How to Start Training: The Effect of Initialization and Architecture
Boris Hanin and David Rolnick · 2018
Later among the works it cites.
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Arthur Jacot, Franck Gabriel, and Clément Hongler · 2018
Later among the works it cites.
Deep Gaussian Processes with Convolutional Kernels
Vinayak Kumar, Vaibhav Singh, PK Srijith, and Andreas Damianou · 2018
Later among the works it cites.
Deep Neural Networks as Gaussian Processes
Jaehoon Lee, Yasaman Bahri, Roman Novak, Sam Schoenholz, Jeffrey Pennington, and Jascha Sohl-dickstein · 2018
Later among the works it cites.
Gaussian Process Behaviour in Wide Deep Neural Networks
Alexander G. de G. Matthews, Jiri Hron, Mark Rowland, Richard E. Turner, and Zoubin Ghahramani · 2018
Later among the works it cites.
Deep learning generalizes because the parameter-function map is biased towards simple functions
Guillermo Valle-Pérez, Chico Q. Camargo, and Ard A. Louis · 2018
Later among the works it cites.
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks
Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel Schoenholz, and Jeffrey Pennington · 2018
Later among the works it cites.
Deep Mean Field Theory: Layerwise Variance and Width Variation as Methods to Control Gradient Explosion
Greg Yang and Sam S. Schoenholz · 2018
Later among the works it cites.
A Mean Field Theory of Batch Normalization
Greg Yang, Jeffrey Pennington, Vinay Rao, Jascha Sohl-Dickstein, and Samuel S. Schoenholz · 2018
Later among the works it cites.
mvtnorm: Multivariate Normal and t Distributions , 2019
Alan Genz, Frank Bretz, Tetsuhisa Miwa, Xuefei Mi, Friedrich Leisch, Fabian Scheipl, and Torsten Hothorn · 2019
Closest in time.
Greg Yang · 2019
Closest in time.
A Fine-Grained Spectral Perspective on Neural Networks
Greg Yang and Hadi Salman · 2019
Closest in time.