A Theoretical Understanding of Neural Network Compression from Sparse Linear Approximation

Understand

The goal of model compression is to reduce the size of a large neural network while retaining a comparable performance.

• As a result, computation and memory costs in resource-limited applications may be significantly reduced by dropping redundant weights, neurons, or layers.
• There have been many model compression algorithms proposed that provide impressive empirical success.
• However, a theoretical understanding of model compression is still limited.