2020

ZeroQ: A Novel Zero Shot Quantization Framework

Cai, Yaohui, Yao, Zhewei, Dong, Zhen et al.

Understand

Quantization is a promising approach for reducing the inference time and memory footprint of neural networks.

However, most existing quantization methods require access to the original training dataset for retraining during quantization.
This is often not possible for applications with sensitive or proprietary data, e.g., due to privacy and security concerns.
Existing zero-shot quantization methods use different heuristics to address this, but they result in poor performance, especially when quantizing to ultra-low precision.

Built on

Experimental determination of precision requirements for back-propagation training of artificial neural networks
Krste Asanovic and Nelson Morgan · 1991
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2014
Earlier work this paper cites.
Computing’s energy problem (and what we can do about it)
Mark Horowitz · 2014
Earlier work this paper cites.
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick · 2014
Earlier work this paper cites.
Fitnets: Hints for thin deep nets
Original
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio · 2014
Earlier work this paper cites.
Binaryconnect: Training deep neural networks with binary weights during propagations
Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David · 2015
Earlier work this paper cites.
Learning both weights and connections for efficient neural network
Song Han, Jeff Pool, John Tran, and William Dally · 2015
Earlier work this paper cites.
Inceptionism: Going deeper into neural networks
Alexander Mordvintsev, Christopher Olah, and Mike Tyka · 2015
Earlier work this paper cites.
Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding
Song Han, Huizi Mao, and William J Dally · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Squeezenet: Alexnet-level accuracy with 50x fewer parameters and
Original
Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer · 2016
Earlier work this paper cites.
Ternary weight networks
Original
Fengfu Li, Bo Zhang, and Bin Liu · 2016
Earlier work this paper cites.
Pruning filters for efficient convnets
Original
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf · 2016
Earlier work this paper cites.
Feature pyramid networks for object detection
Original
Tsung-Yi Lin, Piotr Dollár, Ross B. Girshick, Kaiming He, Bharath Hariharan, and Serge J. Belongie · 2016
Earlier work this paper cites.
Xnor-net: Imagenet classification using binary convolutional neural networks
Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi · 2016
Earlier work this paper cites.
Rethinking the inception architecture for computer vision
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna · 2016
Earlier work this paper cites.

Similar

Quantized convolutional neural networks for mobile devices
Jiaxiang Wu, Cong Leng, Yuhang Wang, Qinghao Hu, and Jian Cheng · 2016
Cited alongside, same era.
Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients
Original
Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou · 2016
Cited alongside, same era.
Mobilenets: Efficient convolutional neural networks for mobile vision applications
Original
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam · 2017
Cited alongside, same era.
Focal loss for dense object detection
Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár · 2017
Cited alongside, same era.
Exploring the regularity of sparse structure in convolutional neural networks
Huizi Mao, Song Han, Jeff Pool, Wenshuo Li, Xingyu Liu, Yu Wang, and William J Dally · 2017
Cited alongside, same era.
Incremental network quantization: Towards lossless cnns with low-precision weights
Aojun Zhou, Anbang Yao, Yiwen Guo, Lin Xu, and Yurong Chen · 2017
Cited alongside, same era.
Trained ternary quantization
Chenzhuo Zhu, Song Han, Huizi Mao, and William J Dally · 2017
Cited alongside, same era.
Post training 4-bit quantization of convolution networks for rapid-deployment
Original
Ron Banner, Yury Nahshan, Elad Hoffer, and Daniel Soudry · 2018
Cited alongside, same era.
Pact: Parameterized clipping activation for quantized neural networks
Original
Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan · 2018
Cited alongside, same era.
Squeezenext: Hardware-aware neural network design
Amir Gholami, Kiseok Kwon, Bichen Wu, Zizheng Tai, Xiangyu Yue, Peter Jin, Sicheng Zhao, and Kurt Keutzer · 2018
Cited alongside, same era.
Quantization and training of neural networks for efficient integer-arithmetic-only inference
Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko · 2018
Cited alongside, same era.
Quantizing deep convolutional networks for efficient inference: A whitepaper
Original
Raghuraman Krishnamoorthi · 2018
Cited alongside, same era.

Then

Mixed precision quantization of convnets via differentiable neural architecture search
Original
Bichen Wu, Yanghan Wang, Peizhao Zhang, Yuandong Tian, Peter Vajda, and Kurt Keutzer · 2018
Later among the works it cites.
LQ-Nets: Learned quantization for highly accurate and compact deep neural networks
Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, and Gang Hua · 2018
Later among the works it cites.
Shufflenet: An extremely efficient convolutional neural network for mobile devices
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun · 2018
Later among the works it cites.
Adaptive quantization for deep neural network
Yiren Zhou, Seyed-Mohsen Moosavi-Dezfooli, Ngai-Man Cheung, and Pascal Frossard · 2018
Later among the works it cites.
https://github.com/amirgholami/zeroq.git, Dec. 2019
2019
Later among the works it cites.
Cat: Compression-aware training for bandwidth reduction
Original
Chaim Baskin, Brian Chmiel, Evgenii Zheltonozhskii, Ron Banner, Alex M Bronstein, and Avi Mendelson · 2019
Later among the works it cites.
Hawq-v2: Hessian aware trace-weighted quantization of neural networks
Original
Zhen Dong, Zhewei Yao, Yaohui Cai, Daiyaan Arfeen, Amir Gholami, Michael W Mahoney, and Kurt Keutzer · 2019
Later among the works it cites.
Hawq: Hessian aware quantization of neural networks with mixed-precision
Zhen Dong, Zhewei Yao, Amir Gholami, Michael Mahoney, and Kurt Keutzer · 2019
Later among the works it cites.
The knowledge within: Methods for data-free model compression
Matan Haroush, Itay Hubara, Elad Hoffer, and Daniel Soudry · 2019
Later among the works it cites.
Low-bit quantization of neural networks for efficient inference
Eli Kravchik, Fan Yang, Pavel Kisilev, and Yoni Choukroun · 2019
Later among the works it cites.
Fully quantized network for object detection
Rundong Li, Yan Wang, Feng Liang, Hongwei Qin, Junjie Yan, and Rui Fan · 2019
Later among the works it cites.
Same, same but different-recovering neural network quantization error through weight factorization
Original
Eldad Meller, Alexander Finkelstein, Uri Almog, and Mark Grobman · 2019
Later among the works it cites.
Data-free quantization through weight equalization and bias correction
Markus Nagel, Mart van Baalen, Tijmen Blankevoort, and Max Welling · 2019
Later among the works it cites.
Q-bert: Hessian based ultra low precision quantization of bert
Original
Sheng Shen, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer · 2019
Later among the works it cites.
HAQ: Hardware-aware automated quantization
Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han · 2019
Later among the works it cites.
Improving neural network quantization without retraining using outlier channel splitting
Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Chris De Sa, and Zhiru Zhang · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…