Evolution of Neural Tangent Kernels under Benign and Adversarial Training

Understand

Two key challenges facing modern deep learning are mitigating deep networks' vulnerability to adversarial attacks and understanding deep learning's generalization capabilities.

• Towards the first issue, many defense strategies have been developed, with the most common being Adversarial Training (AT).
• Towards the second challenge, one of the dominant theories that has emerged is the Neural Tangent Kernel (NTK) -- a characterization of neural network behavior in the infinite-width limit.
• In this limit, the kernel is frozen, and the underlying feature map is fixed.