Fetching the paper…
Reading the bibliography…
Polynomial neural networks (PNNs) have been recently shown to be particularly effective at image generation and face recognition, where high-frequency information is critical.
Frequency principle: Fourier analysis sheds light on deep neural networks
Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, and Zheng Ma · 1901
Earlier work this paper cites.
On the Inductive Bias of Neural Tangent Kernels
Alberto Bietti and Julien Mairal · 1905
Earlier work this paper cites.
The convergence rate of neural networks for learned functions of different frequencies
Ronen Basri, David W. Jacobs, Yoni Kasten, and Shira Kritchman · 1906
Earlier work this paper cites.
Towards Understanding the Spectral Bias of Deep Learning
Yuan Cao, Zhiying Fang, Yue Wu, Ding-Xuan Zhou, and Quanquan Gu · 1912
Earlier work this paper cites.
Polynomial Theory of Complex Systems
A. G. Ivakhnenko · 1971
Earlier work this paper cites.
The pi-sigma network: an efficient higher-order neural network for pattern classification and function approximation
Y. Shin and J. Ghosh · 1991
Earlier work this paper cites.
Convolutional networks for images, speech, and time series
Yann LeCun and Yoshua Bengio · 1998
Earlier work this paper cites.
Regularization with dot-product kernels
Alex Smola, Zoltán Óvári, and Robert C Williamson · 2000
Earlier work this paper cites.
Learning with kernels: support vector machines, regularization, optimization, and beyond
Bernhard Schölkopf, Alexander J Smola, Francis Bach, et al · 2002
Earlier work this paper cites.
Frequency bias in neural networks for input of non-uniform density
Ronen Basri, Meirav Galun, Amnon Geifman, David W. Jacobs, Yoni Kasten, and Shira Kritchman · 2003
Earlier work this paper cites.
A Sigma-Pi-Sigma Neural Network (SPSNN)
Chien-Kuo Li · 2003
Earlier work this paper cites.
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei · 2005
Earlier work this paper cites.
Continuous neural networks
Nicolas Le Roux and Yoshua Bengio · 2007
Earlier work this paper cites.
Kernel Methods for Deep Learning
Youngmin Cho and Lawrence Saul · 2009
Earlier work this paper cites.
Spherical harmonics in p dimensions
Christopher Frye and Costas J Efthimiou · 2012
Earlier work this paper cites.
Sharp analysis of low-rank kernel matrix approximations
Francis R. Bach · 2015
Earlier work this paper cites.
Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber · 2015
Cited alongside, same era.
Breaking the Curse of Dimensionality with Convex Neural Networks
Francis Bach · 2016
Cited alongside, same era.
Neural Machine Translation by Jointly Learning to Align and Translate
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio · 2016
Cited alongside, same era.
A closer look at memorization in deep networks
Devansh Arpit, Stanisław Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Lacoste-Julien · 2017
Cited alongside, same era.
Sort: Second-order response transform for visual recognition
Wide neural networks of any depth evolve as linear models under gradient descent
Jaehoon Lee, Lechao Xiao, Samuel Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington · 2019
Later among the works it cites.
On the Spectral Bias of Neural Networks
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, and Aaron Courville · 2019
Later among the works it cites.
Deep learning generalizes because the parameter-function map is biased towards simple functions
Guillermo Valle-Perez, Chico Q. Camargo, and Ard A. Louis · 2019
Later among the works it cites.
A fine-grained spectral perspective on neural networks
Greg Yang and Hadi Salman · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Yan Wang, Lingxi Xie, Chenxi Liu, Siyuan Qiao, Ya Zhang, Wenjun Zhang, Qi Tian, and Alan Yuille · 2017
Cited alongside, same era.
Understanding deep learning requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals · 2017
Cited alongside, same era.
A convergence theory for deep learning via over-parameterization
Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song · 2018
Cited alongside, same era.
On the power of over-parametrization in neural networks with quadratic activation
Simon Du and Jason Lee · 2018
Cited alongside, same era.
Implicit Bias of Gradient Descent on Linear Convolutional Networks
Suriya Gunasekar, Jason D Lee, Daniel Soudry, and Nati Srebro · 2018
Cited alongside, same era.
Deep Image Prior
Victor Lempitsky, Andrea Vedaldi, and Dmitry Ulyanov · 2018
Cited alongside, same era.
The Implicit Bias of Gradient Descent on Separable Data
Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro · 2018
Cited alongside, same era.
Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks
Sanjeev Arora, Simon Du, Wei Hu, Zhiyuan Li, and Ruosong Wang · 2019
Cited alongside, same era.
Lenaic Chizat, Edouard Oyallon, and Francis Bach · 2020
Later among the works it cites.
P-nets: Deep Polynomial Neural Networks
Grigorios G. Chrysos, Stylianos Moschoglou, Giorgos Bouritsas, Yannis Panagakis, Jiankang Deng, and Stefanos Zafeiriou · 2020
Later among the works it cites.
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Arthur Jacot, Franck Gabriel, and Clément Hongler · 2020
Later among the works it cites.
Multiplicative interactions and where to find them
Siddhant M. Jayakumar, Wojciech M. Czarnecki, Jacob Menick, Jonathan Schwarz, Jack Rae, Simon Osindero, Yee Whye Teh, Tim Harley, and Razvan Pascanu · 2020
Later among the works it cites.
Optimization and generalization of shallow neural networks with quadratic activation functions
Stefano Sarao Mannelli, Eric Vanden-Eijnden, and Lenka Zdeborová · 2020
Later among the works it cites.
Fourier features let networks learn high frequency functions in low dimensional domains
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng · 2020
Later among the works it cites.
Poly-nl: Linear complexity non-local layers with polynomials
Francesca Babiloni, Ioannis Marras, Filippos Kokkinos, Jiankang Deng, Grigorios Chrysos, and Stefanos Zafeiriou · 2021
Later among the works it cites.
Deep Polynomial Neural Networks
Grigorios G. Chrysos, Stylianos Moschoglou, Giorgos Bouritsas, Jiankang Deng, Yannis Panagakis, and Stefanos P Zafeiriou · 2021
Later among the works it cites.
Early-stopped neural networks are consistent
Ziwei Ji, Justin D. Li, and Matus Telgarsky · 2021
Later among the works it cites.
Optimal rates for averaged stochastic gradient descent under neural tangent kernel regime
Atsushi Nitanda and Taiji Suzuki · 2021
Later among the works it cites.
A spectral analysis of dot-product kernels, 2021
Meyer Scetbon and Zaid Harchaoui · 2021
Later among the works it cites.
The product of two ultraspherical polynomials
L. Carlitz · 2040
Closest in time.