Fetching the paper…
Reading the bibliography…
Recent works show an intriguing phenomenon of Frequency Principle (F-Principle) that deep neural networks (DNNs) fit the target function from low to high frequency during the training, which provides insight into the training and generalization behavior of DNNs in complex tasks.
Frequency principle: Fourier analysis sheds light on deep neural networks
Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, and Zheng Ma · 1901
Earlier work this paper cites.
Approximation by superpositions of a sigmoidal function
George Cybenko · 1989
Earlier work this paper cites.
Rademacher and gaussian complexities: Risk bounds and structural results
Peter L Bartlett and Shahar Mendelson · 2002
Earlier work this paper cites.
Understanding machine learning: From theory to algorithms
Shai Shalev-Shwartz and Shai Ben-David · 2014
Earlier work this paper cites.
A closer look at memorization in deep networks
Devansh Arpit, Stanislaw Jastrzbski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al · 2017
Earlier work this paper cites.
Perceptrons: An introduction to computational geometry
Marvin Minsky and Seymour A Papert · 2017
Earlier work this paper cites.
Failures of gradient-based deep learning
Shai Shalev-Shwartz, Ohad Shamir, and Shaked Shammah · 2017
Earlier work this paper cites.
Understanding deep learning requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals · 2017
Earlier work this paper cites.
Understanding training and generalization in deep learning by fourier analysis
Zhiqin John Xu · 2018
Earlier work this paper cites.
Sanjeev Arora, Simon S Du, Wei Hu, Zhiyuan Li, and Ruosong Wang · 2019
Earlier work this paper cites.
The convergence rate of neural networks for learned functions of different frequencies
Ronen Basri, David Jacobs, Yoni Kasten, and Shira Kritchman · 2019
Cited alongside, same era.
Frequency-aware reconstruction of fluid simulations with generative networks
Simon Biland, Vinicius C Azevedo, Byungsoo Kim, and Barbara Solenthaler · 2019
Cited alongside, same era.
A phase shift deep neural network for high frequency approximation and wave problems
Wei Cai, Xiaoguang Li, and Lizuo Liu · 2019
Cited alongside, same era.
Towards Understanding the Spectral Bias of Deep Learning
Yuan Cao, Zhiying Fang, Yue Wu, Ding-Xuan Zhou, and Quanquan Gu · 2019
Cited alongside, same era.
Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
A fine-grained spectral perspective on neural networks
Greg Yang and Hadi Salman · 2019
Later among the works it cites.
Explicitizing an implicit bias of the frequency principle in two-layer neural networks
Yaoyu Zhang, Zhi-Qin John Xu, Tao Luo, and Zheng Ma · 2019
Later among the works it cites.
Science China Mathematics , pages 1–24, 2020
A comparative analysis of the optimization and generalization property of two-layer neural network and random feature models under gradient descent dynamics · 2020
Closest in time.
Frequency bias in neural networks for input of non-uniform density
Ronen Basri, Meirav Galun, Amnon Geifman, David Jacobs, Yoni Kasten, and Shira Kritchman · 2020
Closest in time.
Spectrum dependent learning curves in kernel regression and wide neural networks
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Ameya D. Jagtap, Kenji Kawaguchi, and George Em Karniadakis · 2019
Cited alongside, same era.
Theory of the frequency principle for general deep neural networks
Tao Luo, Zheng Ma, Zhi-Qin John Xu, and Yaoyu Zhang · 2019
Cited alongside, same era.
Neural networks are a priori biased towards boolean functions with low entropy
Chris Mingard, Joar Skalse, Guillermo Valle-Pérez, David Martínez-Rubio, Vladimir Mikulik, and Ard A Louis · 2019
Cited alongside, same era.
Sgd on neural networks learns functions of increasing complexity
Preetum Nakkiran, Gal Kaplun, Dimitris Kalimeris, Tristan Yang, Benjamin L Edelman, Fred Zhang, and Boaz Barak · 2019
Cited alongside, same era.
On the spectral bias of deep neural networks
Nasim Rahaman, Devansh Arpit, Aristide Baratin, Felix Draxler, Min Lin, Fred A Hamprecht, Yoshua Bengio, and Aaron Courville · 2019
Cited alongside, same era.
Deep learning generalizes because the parameter-function map is biased towards simple functions
Guillermo Valle-Perez, Chico Q Camargo, and Ard A Louis · 2019
Cited alongside, same era.
Training behavior of deep neural network in frequency domain
Zhi-Qin J Xu, Yaoyu Zhang, and Yanyang Xiao
Cited in the paper.
Blake Bordelon, Abdulkadir Canatar, and Cengiz Pehlevan · 2020
Closest in time.
Machine learning from a continuous viewpoint, i
Weinan E, Chao Ma, and Lei Wu · 2020
Closest in time.
Ziqi Liu, Wei Cai, and Zhi-Qin John Xu · 2020
Closest in time.
The slow deterioration of the generalization error of the random feature model
Chao Ma, Lei Wu, and Weinan E · 2020
Closest in time.
A type of generalization error induced by initialization in deep neural networks
Yaoyu Zhang, Zhi-Qin John Xu, Tao Luo, and Zheng Ma · 2020
Closest in time.