Bayesian Bits: Unifying Quantization and Pruning

Understand

We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization.

• Bayesian Bits employs a novel decomposition of the quantization operation, which sequentially considers doubling the bit width.
• At each new bit width, the residual error between the full precision value and the previously rounded value is quantized.
• We then decide whether or not to add this quantized residual error for a higher effective bit width and lower quantization noise.