2019

Loss Aware Post-training Quantization

Nahshan, Yury, Chmiel, Brian, Baskin, Chaim et al.

Understand

Neural network quantization enables the deployment of large models on resource-constrained devices.

Current post-training quantization methods fall short in terms of accuracy for INT4 (or lower) but provide reasonable accuracy for INT8 (or above).
In this work, we study the effect of quantization on the structure of the loss landscape.
Additionally, we show that the structure is flat and separable for mild quantization, enabling straightforward post-training quantization methods to achieve good results.

Built on

Zhao, R., Hu, Y., Dotzel, J., De Sa, C., Zhang, Z.: Improving neural network quantization using outlier channel splitting. arXiv preprint arXiv:1901.09504 (2019), URL https://arxiv.org/abs/1901.09504
Original
1901
Earlier work this paper cites.
Bethge, J., Yang, H., Bornstein, M., Meinel, C.: Back to simplicity: How to train accurate bnns from scratch? arXiv preprint arXiv:1906.08637 (2019), URL http://arxiv.org/abs/1906.08637
Original
1906
Earlier work this paper cites.
Finkelstein, A., Almog, U., Grobman, M.: Fighting quantization bias with bias. arXiv preprint arXiv:1906.03193 (2019), URL http://arxiv.org/abs/1906.03193
Original
1906
Earlier work this paper cites.
Nayak, P., Zhang, D., Chai, S.: Bit efficient quantization for deep neural networks. arXiv preprint arXiv:1910.04877 (2019), URL http://arxiv.org/abs/1910.04877
Original
1910
Earlier work this paper cites.
Jin, Q., Yang, L., Liao, Z.: Towards efficient training for neural network quantization. arXiv preprint arXiv:1912.10207 (2019), URL https://arxiv.org/abs/1912.10207
Original
1912
Earlier work this paper cites.
Powell, M.J.: An efficient method for finding the minimum of a function of several variables without calculating derivatives. The Computer Journal 7
1964
Earlier work this paper cites.
Fang, J., Shafiee, A., Abdel-Aziz, H., Thorsley, D., Georgiadis, G., Hassoun, J.: Near-lossless post-training quantization of deep neural networks via a piecewise linear approximation. arXiv preprint arXiv:2002.00104 (2020), URL https://arxiv.org/abs/2002.00104
Original
2002
Earlier work this paper cites.
Kim, H., Kim, K., Kim, J., Kim, J.J.: Binaryduo: Reducing gradient mismatch in binary activation network by coupling binary activations. arXiv preprint arXiv:2002.06517 (2020), URL https://arxiv.org/abs/2002.06517
Original
2002
Earlier work this paper cites.
Goldman, R.: Curvature formulas for implicit curves and surfaces. Computer Aided Geometric Design 22
2005
Earlier work this paper cites.

Similar

He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.S.: Neural collaborative filtering. In: Proceedings of the 26th International Conference on World Wide Web, pp. 173–182, WWW ’17, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland (2017), ISBN 978-1-4503-4913-0, doi: 10.1145/3038912.3052569
2017
Cited alongside, same era.
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., Bengio, Y.: Quantized neural networks: Training neural networks with low precision weights and activations. J. Mach. Learn. Res. 18
2017
Cited alongside, same era.
Mellempudi, N., Kundu, A., Mudigere, D., Das, D., Kaul, B., Dubey, P.: Ternary neural networks with fine-grained quantization. arXiv preprint arXiv:1705.01462 (2017), URL http://arxiv.org/abs/1705.01462
Original
2017
Cited alongside, same era.
Migacz, S.: 8-bit inference with tensorrt (2017), URL http://on-demand.gputechconf.com/gtc/2017/presentation/s7310-8-bit-inference-with-tensorrt.pdf
2017
Cited alongside, same era.
Banner, R., Nahshan, Y., Hoffer, E., Soudry, D.: Post-training 4-bit quantization of convolution networks for rapid-deployment. arXiv preprint arXiv:1810.05723 (2018), URL http://arxiv.org/abs/1810.05723
Original
2018
Cited alongside, same era.
Baskin, C., Liss, N., Chai, Y., Zheltonozhskii, E., Schwartz, E., Girayes, R., Mendelson, A., Bronstein, A.M.: Nice: Noise injection and clamping estimation for neural network quantization. arXiv preprint arXiv:1810.00162 (2018), URL http://arxiv.org/abs/1810.00162
Original
2018
Cited alongside, same era.
Faraone, J., Fraser, N., Blott, M., Leong, P.H.: Syq: Learning symmetric quantization for efficient deep neural networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2018), URL http://openaccess.thecvf.com/content_cvpr_2018/html/Faraone_SYQ_Learning_Symmetric_CVPR_2018_paper.html
2018
Cited alongside, same era.
Gong, J., Shen, H., Zhang, G., Liu, X., Li, S., Jin, G., Maheshwari, N., Fomenko, E., Segal, E.: Highly efficient 8-bit low precision inference of convolutional neural networks with intelcaffe. In: Proceedings of the 1st on Reproducible Quality-Efficient Systems Tournament on Co-designing Pareto-efficient Deep Learning, ReQuEST ’18, ACM, New York, NY, USA (2018), ISBN 978-1-4503-5923-8, doi: 10.1145/3229762.3229763
2018
Cited alongside, same era.
Krishnamoorthi, R.: Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342 (2018), URL http://arxiv.org/abs/1806.08342
Original
2018
Cited alongside, same era.

Then

Lee, J.H., Ha, S., Choi, S., Lee, W.J., Lee, S.: Quantization for rapid deployment of deep neural networks. arXiv preprint arXiv:1810.05488 (2018), URL http://arxiv.org/abs/1810.05488
Original
2018
Later among the works it cites.
Zhang, D., Yang, J., Ye, D., Hua, G.: Lq-nets: Learned quantization for highly accurate and compact deep neural networks. In: The European Conference on Computer Vision (ECCV) (September 2018), URL http://openaccess.thecvf.com/content_ECCV_2018/html/Dongqing_Zhang_Optimized_Quantization_for_ECCV_2018_paper.html
2018
Later among the works it cites.
Gong, R., Liu, X., Jiang, S., Li, T., Hu, P., Lin, J., Yu, F., Yan, J.: Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In: The IEEE International Conference on Computer Vision (ICCV) (October 2019), URL http://openaccess.thecvf.com/content_ICCV_2019/html/Gong_Differentiable_Soft_Quantization_Bridging_Full-Precision_and_Low-Bit_Neural_Networks_ICCV_2019_paper.html
2019
Closest in time.
Kravchik, E., Yang, F., Kisilev, P., Choukroun, Y.: Low-bit quantization of neural networks for efficient inference. In: The IEEE International Conference on Computer Vision (ICCV) Workshops (Oct 2019), URL http://openaccess.thecvf.com/content_ICCVW_2019/html/CEFRL/Kravchik_Low-bit_Quantization_of_Neural_Networks_for_Efficient_Inference_ICCVW_2019_paper.html
2019
Closest in time.
Liu, C., Ding, W., Xia, X., Hu, Y., Zhang, B., Liu, J., Zhuang, B., Guo, G.: Rectified binary convolutional networks for enhancing the performance of 1-bit DCNNs. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pp. 854–860, International Joint Conferences on Artificial Intelligence Organization (7 2019), doi: 10.24963/ijcai.2019/120
2019
Closest in time.
Nagel, M., Baalen, M.v., Blankevoort, T., Welling, M.: Data-free quantization through weight equalization and bias correction. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1325–1334 (2019), URL http://openaccess.thecvf.com/content_ICCV_2019/html/Nagel_Data-Free_Quantization_Through_Weight_Equalization_and_Bias_Correction_ICCV_2019_paper.html
2019
Closest in time.
Peng, H., Chen, S.: Bdnn: Binary convolution neural networks for fast object detection. Pattern Recognition Letters (2019), ISSN 0167-8655, doi: https://doi.org/10.1016/j.patrec.2019.03.026
2019
Closest in time.
Yang, J., Shen, X., Xing, J., Tian, X., Li, H., Deng, B., Huang, J., Hua, X.s.: Quantization networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019), URL http://openaccess.thecvf.com/content_CVPR_2019/html/Yang_Quantization_Networks_CVPR_2019_paper.html
2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…