Mixed Low-precision Deep Learning Inference using Dynamic Fixed Point

Understand

We propose a cluster-based quantization method to convert pre-trained full precision weights into ternary weights with minimal impact on the accuracy.

• In addition, we also constrain the activations to 8-bits thus enabling sub 8-bit full integer inference pipeline.
• Our method uses smaller clusters of N filters with a common scaling factor to minimize the quantization loss, while also maximizing the number of ternary operations.
• We show that with a cluster size of N=4 on Resnet-101, can achieve 71.8% TOP-1 accuracy, within 6% of the best full precision results while replacing ~85% of all multiplications with 8-bit accumulations.