Mixed Precision DNNs: All you need is a good parametrization

Understand

Efficient deep neural network (DNN) inference on mobile or embedded devices typically involves quantization of the network parameters and activations.

• In particular, mixed precision networks achieve better performance than networks with homogeneous bitwidth for the same size constraint.
• Since choosing the optimal bitwidths is not straight forward, training methods, which can learn them, are desirable.
• Differentiable quantization with straight-through gradients allows to learn the quantizer's parameters using gradient methods.