HAQ: Hardware-Aware Automated Quantization with Mixed Precision

Understand

Model quantization is a widely used technique to compress and accelerate deep neural network (DNN) inference.

• Emergent DNN hardware accelerators begin to support mixed precision (1-8 bits) to further improve the computation efficiency, which raises a great challenge to find the optimal bitwidth for each layer: it requires domain experts to explore the vast design space trading off among accuracy, latency, energy, and model size, which is both time-consuming and sub-optimal.
• Conventional quantization algorithm ignores the different hardware architectures and quantizes all the layers in a uniform way.
• In this paper, we introduce the Hardware-Aware Automated Quantization (HAQ) framework which leverages the reinforcement learning to automatically determine the quantization policy, and we take the hardware accelerator's feedback in the design loop.