Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorization

Understand

Quantization of neural networks has become common practice, driven by the need for efficient implementations of deep neural networks on embedded devices.

• In this paper, we exploit an oft-overlooked degree of freedom in most networks - for a given layer, individual output channels can be scaled by any factor provided that the corresponding weights of the next layer are inversely scaled.
• Therefore, a given network has many factorizations which change the weights of the network without changing its function.
• We present a conceptually simple and easy to implement method that uses this property and show that proper factorizations significantly decrease the degradation caused by quantization.