Understanding Generalization in Deep Learning via Tensor Methods

Understand

Deep neural networks generalize well on unseen data though the number of parameters often far exceeds the number of training examples.

• Recently proposed complexity measures have provided insights to understanding the generalizability in neural networks from perspectives of PAC-Bayes, robustness, overparametrization, compression and so on.
• In this work, we advance the understanding of the relations between the network's architecture and its generalizability from the compression perspective.
• Using tensor analysis, we propose a series of intuitive, data-dependent and easily-measurable properties that tightly characterize the compressibility and generalizability of neural networks; thus, in practice, our generalization bound outperforms the previous compression-based ones, especially for neural networks using tensors as their weight kernels (e.g.