Improving Neural Network Quantization without Retraining using Outlier Channel Splitting

Understand

Quantization can improve the execution latency and energy efficiency of neural networks on both commodity GPUs and specialized accelerators.

• The majority of existing literature focuses on training quantized DNNs, while this work examines the less-studied topic of quantizing a floating-point model without (re)training.
• DNN weights and activations follow a bell-shaped distribution post-training, while practical hardware uses a linear quantization grid.
• This leads to challenges in dealing with outliers in the distribution.