Fetching the paper…
Reading the bibliography…
Infinite width limit has shed light on generalization and optimization aspects of deep learning by establishing connections between neural networks and kernel methods.
Calculation of Gauss quadrature rules
Gene H Golub and John H Welsch · 1969
Earlier work this paper cites.
Handbook of mathematical functions with formulas, graphs, and mathematical tables , 1988
Milton Abramowitz, Irene A Stegun, and Robert H Romer · 1988
Earlier work this paper cites.
Priors for infinite networks
Radford M. Neal · 1994
Earlier work this paper cites.
Computing with infinite networks
Christopher Williams · 1996
Earlier work this paper cites.
A note on multivariate Gauss-Hermite quadrature
Peter Jäckel · 2005
Earlier work this paper cites.
Kernel methods for deep learning
Youngmin Cho and Lawrence Saul · 2009
Earlier work this paper cites.
Random Features for Large-Scale Kernel Machines
Ali Rahimi and Benjamin Recht · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky · 2009
Earlier work this paper cites.
Asymptotics on Laguerre or Hermite polynomial expansions and their applications in Gauss quadrature
Shuhuang Xiang · 2012
Earlier work this paper cites.
Probability theory: a comprehensive course
Achim Klenke · 2013
Earlier work this paper cites.
Rectifier nonlinearities improve neural network acoustic models
Andrew L Maas, Awni Y Hannun, Andrew Y Ng, et al · 2013
Earlier work this paper cites.
Julien Mairal, Piotr Koniusz, Zaid Harchaoui, and Cordelia Schmid · 2014
Earlier work this paper cites.
Ryan O’Donnell · 2014
Earlier work this paper cites.
Gaussian error linear units (gelus)
Dan Hendrycks and Kevin Gimpel · 2016
Earlier work this paper cites.
Amit Daniely, Roy Frostig, and Yoram Singer · 2016
Earlier work this paper cites.
Fast and accurate deep network learning by exponential linear units (elus)
Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter · 2016
Earlier work this paper cites.
Deep Neural Networks as Gaussian Processes
Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S Schoenholz, Jeffrey Pennington, and Jascha Sohl-Dickstein · 2018
Earlier work this paper cites.
Gaussian Process Behaviour in Wide Deep Neural Networks
Alexander G. de G. Matthews, Jiri Hron, Mark Rowland, Richard E. Turner, and Zoubin Ghahramani · 2018
Earlier work this paper cites.
Neural tangent kernel: Convergence and generalization in neural networks
Arthur Jacot, Franck Gabriel, and Clément Hongler · 2018
Earlier work this paper cites.
On the optimization of deep networks: Implicit acceleration by overparameterization
Sanjeev Arora, Nadav Cohen, and Elad Hazan · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Earlier work this paper cites.
A random matrix approach to neural networks
Cosme Louart, Zhenyu Liao, Romain Couillet, et al · 2018
Earlier work this paper cites.
Invariance of weight distributions in rectified MLPs
Russell Tsuchida, Fred Roosta, and Marcus Gallagher · 2018
Cited alongside, same era.
Wide neural networks of any depth evolve as linear models under gradient descent
Jaehoon Lee, Lechao Xiao, Samuel Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington · 2019
Cited alongside, same era.
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes
Roman Novak, Lechao Xiao, Jaehoon Lee, Yasaman Bahri, Greg Yang, Jiri Hron, Daniel A. Abolafia, Jeffrey Pennington, and Jascha Sohl-Dickstein · 2019
Cited alongside, same era.
Deep convolutional networks as shallow Gaussian processes
Adrià Garriga-Alonso, Laurence Aitchison, and Carl Edward Rasmussen · 2019
Cited alongside, same era.
Cihang Xie, Mingxing Tan, Boqing Gong, Alan Yuille, and Quoc V Le · 2020
Later among the works it cites.
Neural Tangents: Fast and Easy Infinite Neural Networks in Python
Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, and Samuel S. Schoenholz · 2020
Later among the works it cites.
Expressive priors in Bayesian neural networks: Kernel combinations and periodic functions
Tim Pearce, Russell Tsuchida, Mohamed Zaki, Alexandra Brintrup, and Andy Neely · 2020
Later among the works it cites.
Neural kernels without tangents
Vaishaal Shankar, Alex Fang, Wenshuo Guo, Sara Fridovich-Keil, Jonathan Ragan-Kelley, Ludwig Schmidt, and Benjamin Recht · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Simon S Du, Kangcheng Hou, Russ R Salakhutdinov, Barnabas Poczos, Ruosong Wang, and Keyulu Xu · 2019
Cited alongside, same era.
Greg Yang · 2019
Cited alongside, same era.
A convergence theory for deep learning via over-parameterization
Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song · 2019
Cited alongside, same era.
The role of over-parametrization in generalization of neural networks
Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, and Nathan Srebro · 2019
Cited alongside, same era.
Generalization bounds of stochastic gradient descent for wide and deep neural networks
Yuan Cao and Quanquan Gu · 2019
Cited alongside, same era.
Regularization matters: Generalization and optimization of neural nets vs their induced kernel
Colin Wei, Jason D Lee, Qiang Liu, and Tengyu Ma · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever · 2019
Cited alongside, same era.
Richer priors for infinitely wide multi-layer perceptrons
Russell Tsuchida, Fred Roosta, and Marcus Gallagher · 2019
Cited alongside, same era.
Thomas D Ahle, Michael Kapralov, Jakob BT Knudsen, Rasmus Pagh, Ameya Velingker, David P Woodruff, and Amir Zandieh · 2020
Later among the works it cites.
Finite versus infinite neural networks: an empirical study
Jaehoon Lee, Samuel Schoenholz, Jeffrey Pennington, Ben Adlam, Lechao Xiao, Roman Novak, and Jascha Sohl-Dickstein · 2020
Later among the works it cites.
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors · 2020
Later among the works it cites.
Albert: A lite bert for self-supervised learning of language representations
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut · 2020
Later among the works it cites.
Elvis Dohmatob · 2021
Later among the works it cites.
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective
Wuyang Chen, Xinyu Gong, and Zhangyang Wang · 2021
Later among the works it cites.
Fl-ntk: A neural tangent kernel-based framework for federated learning analysis
Baihe Huang, Xiaoxiao Li, Zhao Song, and Xin Yang · 2021
Later among the works it cites.
Meta-learning with neural tangent kernels
Yufan Zhou, Zhenyi Wang, Jiayi Xian, Changyou Chen, and Jinhui Xu · 2021
Later among the works it cites.
Neural tangent generalization attacks
Chia-Hung Yuan and Shan-Hung Wu · 2021
Later among the works it cites.
Scaling Neural Tangent Kernels via Sketching and Random Features
Amir Zandieh, Insu Han, Haim Avron, Neta Shoham, Chaewon Kim, and Jinwoo Shin · 2021
Later among the works it cites.
Avoiding Kernel Fixed Points: Computing with ELU and GELU Infinite Networks
Russell Tsuchida, Tim Pearce, Chris van der Heide, Fred Roosta, and Marcus Gallagher · 2021
Later among the works it cites.
Reverse Engineering the Neural Tangent Kernel
James B Simon, Sajant Anand, and Michael R DeWeese · 2021
Later among the works it cites.
Exploring the Uncertainty Properties of Neural Networks’ Implicit Priors in the Infinite-Width Limit
Ben Adlam, Jaehoon Lee, Lechao Xiao, Jeffrey Pennington, and Jasper Snoek · 2021
Later among the works it cites.
Eigenspace restructuring: a principle of space and frequency in neural networks
Lechao Xiao · 2022
Closest in time.
Hamed Hassani and Adel Javanmard · 2022
Closest in time.
Fast Finite Width Neural Tangent Kernel
Roman Novak, Jascha Sohl-Dickstein, and Samuel S. Schoenholz · 2022
Closest in time.
A random matrix perspective on mixtures of nonlinearities in high dimensions
Ben Adlam, Jake Levinson, and Jeffrey Pennington · 2022
Closest in time.