Fetching the paper…
Reading the bibliography…
Several recent trends in machine learning theory and practice, from the design of state-of-the-art Gaussian Process to the convergence analysis of deep neural nets (DNNs) under stochastic gradient descent (SGD), have found it fruitful to study wide random neural networks.
A Generalization Theory of Gradient Descent for Learning Over-parameterized Deep ReLU Networks
Cao, Y. and Gu, Q · 1902
Earlier work this paper cites.
Spin-glass models of neural networks
Amit, D. J., Gutfreund, H., and Sompolinsky, H · 1985
Earlier work this paper cites.
Chaos in Random Neural Networks
Sompolinsky, H., Crisanti, A., and Sommers, H. J · 1988
Earlier work this paper cites.
BAYESIAN LEARNING FOR NEURAL NETWORKS
Neal, R. M · 1995
Earlier work this paper cites.
Long Short-Term Memory
Hochreiter, S. and Schmidhuber, J · 1997
Earlier work this paper cites.
On the strong law for arrays and for the bootstrap mean and variance
Hu, T.-C. and L. Taylor, R · 1997
Earlier work this paper cites.
Computing with Infinite Networks
Williams, C. K. I · 1997
Earlier work this paper cites.
Gradient-based learning applied to document recognition
LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P · 1998
Earlier work this paper cites.
Object recognition with gradient-based learning
LeCun, Y., Haffner, P., Bottou, L., and Bengio, Y · 1999
Earlier work this paper cites.
A limit theorem at the edge of a non-Hermitian random matrix ensemble
Rider, B · 2003
Earlier work this paper cites.
Hierarchical Gaussian process latent variable models
Lawrence, N. D. and Moore, A. J · 2007
Earlier work this paper cites.
Continuous neural networks
Le Roux, N. and Bengio, Y · 2007
Earlier work this paper cites.
Kernel methods for deep learning
Cho, Y. and Saul, L. K · 2009
Earlier work this paper cites.
Message Passing Algorithms for Compressed Sensing
Donoho, D. L., Maleki, A., and Montanari, A · 2009
Earlier work this paper cites.
The dynamics of message passing on dense graphs, with applications to compressed sensing
Bayati, M. and Montanari, A · 2010
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Glorot, X. and Bengio, Y · 2010
Earlier work this paper cites.
Stimulus-Dependent Suppression of Chaos in Recurrent Neural Networks
Rajan, K., Abbott, L. F., and Sompolinsky, H · 2010
Earlier work this paper cites.
An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model
Bolthausen, E · 2012
Earlier work this paper cites.
Approximate Message Passing with Consistent Parameter Estimation and Applications to Sparse Learning
Kamilov, U. S., Rangan, S., Fletcher, A. K., and Unser, M · 2012
Earlier work this paper cites.
Topics in random matrix theory
Tao, T · 2012
Earlier work this paper cites.
Deep gaussian processes
Damianou, A. and Lawrence, N · 2013
Earlier work this paper cites.
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., and Bengio, Y · 2014
Earlier work this paper cites.
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y · 2014
Earlier work this paper cites.
Analysis of boolean functions
O’Donnell, R · 2014
Earlier work this paper cites.
Compressive Phase Retrieval via Generalized Approximate Message Passing
Schniter, P. and Rangan, S · 2014
Earlier work this paper cites.
Dynamics of Random Neural Networks with Bistable Units
Stern, M., Sompolinsky, H., and Abbott, L. F · 2014
Cited alongside, same era.
Steps Toward Deep Kernel Methods from Infinite Neural Networks
Hazan, T. and Jaakkola, T · 2015
Cited alongside, same era.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
He, K., Zhang, X., Ren, S., and Sun, J · 2015
Cited alongside, same era.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. and Szegedy, C · 2015
Cited alongside, same era.
Transition to Chaos in Random Neuronal Networks
Kadmon, J. and Sompolinsky, H · 2015
Cited alongside, same era.
A gaussian process perspective on convolutional neural networks
Borovykh, A · 2018
Later among the works it cites.
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks
Chen, M., Pennington, J., and Schoenholz, S · 2018
Later among the works it cites.
Path Integral Approach to Random Neural Networks
Crisanti, A. and Sompolinsky, H · 2018
Later among the works it cites.
Entropy and mutual information in models of deep neural networks
Gabrié, M., Manoel, A., Luneau, C., Barbier, J., Macris, N., Krzakala, F., and Zdeborová, L · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Ba, J. L., Kiros, J. R., and Hinton, G. E · 2016
Cited alongside, same era.
Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity
Daniely, A., Frostig, R., and Singer, Y · 2016
Cited alongside, same era.
Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?
Giryes, R., Sapiro, G., and Bronstein, A. M · 2016
Cited alongside, same era.
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Cited alongside, same era.
Densely Connected Convolutional Networks
Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K. Q · 2016
Cited alongside, same era.
Phase Transitions and Sample Complexity in Bayes-Optimal Matrix Factorization
Kabashima, Y., Krzakala, F., Mézard, M., Sakata, A., and Zdeborová, L · 2016
Cited alongside, same era.
Exponential expressivity in deep neural networks through transient chaos
Poole, B., Lahiri, S., Raghu, M., Sohl-Dickstein, J., and Ganguli, S · 2016
Cited alongside, same era.
Garriga-Alonso, A., Aitchison, L., and Rasmussen, C. E · 2018
Later among the works it cites.
How to Start Training: The Effect of Initialization and Architecture
Hanin, B. and Rolnick, D · 2018
Later among the works it cites.
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Jacot, A., Gabriel, F., and Hongler, C · 2018
Later among the works it cites.
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach
Karakida, R., Akaho, S., and Amari, S.-i · 2018
Later among the works it cites.
Deep Gaussian Processes with Convolutional Kernels
Kumar, V., Singh, V., Srijith, P., and Damianou, A · 2018
Later among the works it cites.
Coherent chaos in a recurrent neural network with structured connectivity
Landau, I. D. and Sompolinsky, H · 2018
Later among the works it cites.
Deep Neural Networks as Gaussian Processes
Lee, J., Bahri, Y., Novak, R., Schoenholz, S., Pennington, J., and Sohl-dickstein, J · 2018
Later among the works it cites.
Exploring the Function Space of Deep-Learning Machines
Li, B. and Saad, D · 2018
Later among the works it cites.
On Random Deep Weight-Tied Autoencoders: Exact Asymptotic Analysis, Phase Transitions, and Implications to Training
Li, P. and Nguyen, P.-M · 2018
Later among the works it cites.
Gaussian Process Behaviour in Wide Deep Neural Networks
Matthews, A. G. d. G., Rowland, M., Hron, J., Turner, R. E., and Ghahramani, Z · 2018
Later among the works it cites.
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes
Novak, R., Xiao, L., Lee, J., Bahri, Y., Abolafia, D. A., Pennington, J., and Sohl-Dickstein, J · 2018
Later among the works it cites.
Randomized Prior Functions for Deep Reinforcement Learning
Osband, I., Aslanides, J., and Cassirer, A · 2018
Later among the works it cites.
The Spectrum of the Fisher Information Matrix of a Single-Hidden-Layer Neural Network
Pennington, J. and Worah, P · 2018
Later among the works it cites.
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks
Xiao, L., Bahri, Y., Sohl-Dickstein, J., Schoenholz, S., and Pennington, J · 2018
Later among the works it cites.
Deep Mean Field Theory: Layerwise Variance and Width Variation as Methods to Control Gradient Explosion
Yang, G. and Schoenholz, S. S · 2018
Later among the works it cites.
A Mean Field Theory of Batch Normalization
Yang, G., Pennington, J., Rao, V., Sohl-Dickstein, J., and Schoenholz, S. S · 2018
Later among the works it cites.
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks
Zou, D., Cao, Y., Zhou, D., and Gu, Q · 2018
Later among the works it cites.
Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes
Yang, G · 2019
Closest in time.
Asymptotic mutual information for the balanced binary stochastic block model
Deshpande, Y., Abbe, E., and Montanari, A · 2049
Closest in time.
High dimensional robust M-estimation: asymptotic variance via approximate message passing
Donoho, D. and Montanari, A · 2064
Closest in time.