Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network

Understand

One of the biggest issues in deep learning theory is the generalization ability of networks with huge model size.

• The classical learning theory suggests that overparameterized models cause overfitting.
• However, practically used large deep models avoid overfitting, which is not well explained by the classical approaches.
• To resolve this issue, several attempts have been made.