A Unified Framework of DNN Weight Pruning and Weight Clustering/Quantization Using ADMM

Understand

Many model compression techniques of Deep Neural Networks (DNNs) have been investigated, including weight pruning, weight clustering and quantization, etc.

• Weight pruning leverages the redundancy in the number of weights in DNNs, while weight clustering/quantization leverages the redundancy in the number of bit representations of weights.
• They can be effectively combined in order to exploit the maximum degree of redundancy.
• However, there lacks a systematic investigation in literature towards this direction.