Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)

Understand

Deep learning algorithms achieve high classification accuracy at the expense of significant computation cost.

• In order to reduce this cost, several quantization schemes have gained attention recently with some focusing on weight quantization, and others focusing on quantizing activations.
• This paper proposes novel techniques that target weight and activation quantizations separately resulting in an overall quantized neural network (QNN).
• The activation quantization technique, PArameterized Clipping acTivation (PACT), uses an activation clipping parameter $\alpha$ that is optimized during training to find the right quantization scale.