Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks

Understand

Understanding the fundamental principles behind the massive success of neural networks is one of the most important open questions in deep learning.

• However, due to the highly complex nature of the problem, progress has been relatively slow.
• In this note, through the lens of infinite-width networks, a.k.a.
• neural kernels, we present one such principle resulting from hierarchical localities.