Spectral Pruning: Compressing Deep Neural Networks via Spectral Analysis and its Generalization Error

Understand

Compression techniques for deep neural network models are becoming very important for the efficient execution of high-performance deep learning systems on edge-computing devices.

• The concept of model compression is also important for analyzing the generalization error of deep learning, known as the compression-based error bound.
• However, there is still huge gap between a practically effective compression method and its rigorous background of statistical learning theory.
• To resolve this issue, we develop a new theoretical framework for model compression and propose a new pruning method called {\it spectral pruning} based on this framework.