Built on

H. J. Kelley, “Gradient theory of optimal flight paths,” Ars Journal , vol. 30, no. 10, pp. 947–954, 1960
1960
Earlier work this paper cites.
Y. L. Cun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” in Advances in Neural Information Processing Systems . Morgan Kaufmann, 1990, pp. 598–605
1990
Earlier work this paper cites.
B. Hassibi and D. G. Stork, “Second order derivatives for network pruning: Optimal brain surgeon,” in NIPS , 1992
1992
Earlier work this paper cites.
F. Mireshghallah, M. Taram, P. Vepakomma, A. Singh, R. Raskar, and H. Esmaeilzadeh, “Privacy in deep learning: A survey,” CoRR , vol. abs/2004.12254, 2020. [Online]. Available: https://arxiv.org/abs/2004.12254
Original
2004
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” in CVPR09 , 2009
2009
Earlier work this paper cites.
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 , ser. NIPS’14. Cambridge, MA, USA: MIT Press, 2014, p. 2672–2680
2014
Earlier work this paper cites.
S. Han, J. Pool, J. Tran, and W. J. Dally, “Learning both weights and connections for efficient neural network,” ArXiv , vol. abs/1506.02626, 2015
Original
2015
Earlier work this paper cites.
G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” in NIPS Deep Learning and Representation Learning Workshop , 2015. [Online]. Available: http://arxiv.org/abs/1503.02531
Original
2015
Earlier work this paper cites.
S. Srinivas and R. V. Babu, “Data-free parameter pruning for deep neural networks,” in BMVC , 2015
2015
Earlier work this paper cites.
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 , ser. ICML’15. JMLR.org, 2015, p. 448–456
2015
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , Y. Bengio and Y. LeCun, Eds., 2015. [Online]. Available: http://arxiv.org/abs/1412.6980
Original
2015
Earlier work this paper cites.
M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi, “Xnor-net: Imagenet classification using binary convolutional neural networks,” in European Conference on Computer Vision . Springer, 2016, pp. 525–542
2016
Earlier work this paper cites.
H. B. McMahan, E. Moore, D. Ramage, and B. A. y Arcas, “Federated learning of deep networks using model averaging,” CoRR , vol. abs/1602.05629, 2016. [Online]. Available: http://arxiv.org/abs/1602.05629
Original
2016
Earlier work this paper cites.
M. Courbariaux and Y. Bengio, “Binarynet: Training deep neural networks with weights and activations constrained to +1 or -1,” CoRR , vol. abs/1602.02830, 2016. [Online]. Available: http://arxiv.org/abs/1602.02830
Original
2016
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2016
2016
Earlier work this paper cites.
A. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” ArXiv , vol. abs/1704.04861, 2017
Original
2017
Earlier work this paper cites.
J. Redmon and A. Farhadi, “Yolo9000: Better, faster, stronger,” in Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on . IEEE, 2017, pp. 6517–6525
2017
Earlier work this paper cites.
X. Dong, S. Chen, and S. J. Pan, “Learning to prune deep neural networks via layer-wise optimal brain surgeon,” in Proceedings of the 31st International Conference on Neural Information Processing Systems , ser. NIPS’17. Red Hook, NY, USA: Curran Associates Inc., 2017, p. 4860–4874
2017
Earlier work this paper cites.
B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018
2018
Earlier work this paper cites.

Similar

R. Krishnamoorthi, “Quantizing deep convolutional networks for efficient inference: A whitepaper,” CoRR , vol. abs/1806.08342, 2018. [Online]. Available: http://arxiv.org/abs/1806.08342
Original
2018
Cited alongside, same era.
B. Liu, Y. Cao, M. Long, J. Wang, and J. Wang, “Deep triplet quantization,” in Proceedings of the 26th ACM International Conference on Multimedia , ser. MM ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 755–763. [Online]. Available: https://doi.org/10.1145/3240508.3240516
2018
Cited alongside, same era.
R. Arora, A. Basu, P. Mianjy, and A. Mukherjee, “Understanding deep neural networks with rectified linear units,” in International Conference on Learning Representations , 2018. [Online]. Available: https://openreview.net/forum?id=B1J_rgWRW
2018
Cited alongside, same era.
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018

Then

H. Wu, P. Judd, X. Zhang, M. Isaev, and P. Micikevicius, “Integer quantization for deep learning inference: Principles and empirical evaluation,” ArXiv , vol. abs/2004.09602, 2020
Original
2020
Closest in time.
Y. Yang, S. Wu, L. Deng, T. Yan, Y. Xie, and G. Li, “Training high-performance and large-scale deep neural networks with full 8-bit integers,” Neural networks : the official journal of the International Neural Network Society , vol. 125, pp. 70–82, 2020
2020
Closest in time.
G. Di Guglielmo, J. M. Duarte, P. Harris, D. Hoang, S. Jindariani, E. Kreinar, M. Liu, V. Loncar, J. Ngadiuba, K. Pedro, and et al., “Compressing deep neural networks on fpgas to binary and ternary precision with hls4ml,” Machine Learning: Science and Technology , Jun 2020. [Online]. Available: http://dx.doi.org/10.1088/2632-2153/aba042
2020
Closest in time.
B. Martinez, J. Yang, A. Bulat, and G. Tzimiropoulos, “Training binary neural networks with real-to-binary convolutions,” in International Conference on Learning Representations , 2020. [Online]. Available: https://openreview.net/forum?id=BJg4NgBKvH

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2020

Layer-Wise Data-Free CNN Compression

Horton, Maxwell, Jin, Yanzi, Farhadi, Ali et al.

Understand

We present a computationally efficient method for compressing a trained neural network without using real data.

We break the problem of data-free network compression into independent layer-wise compressions.
We show how to efficiently generate layer-wise training data using only a pretrained network.
We use this data to perform independent layer-wise compressions on the pretrained network.

Built on

H. J. Kelley, “Gradient theory of optimal flight paths,” Ars Journal , vol. 30, no. 10, pp. 947–954, 1960
1960
Earlier work this paper cites.
Y. L. Cun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” in Advances in Neural Information Processing Systems . Morgan Kaufmann, 1990, pp. 598–605
1990
Earlier work this paper cites.
B. Hassibi and D. G. Stork, “Second order derivatives for network pruning: Optimal brain surgeon,” in NIPS , 1992
1992
Earlier work this paper cites.
F. Mireshghallah, M. Taram, P. Vepakomma, A. Singh, R. Raskar, and H. Esmaeilzadeh, “Privacy in deep learning: A survey,” CoRR , vol. abs/2004.12254, 2020. [Online]. Available: https://arxiv.org/abs/2004.12254
Original
2004
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” in CVPR09 , 2009
2009
Earlier work this paper cites.
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 , ser. NIPS’14. Cambridge, MA, USA: MIT Press, 2014, p. 2672–2680
2014
Earlier work this paper cites.
S. Han, J. Pool, J. Tran, and W. J. Dally, “Learning both weights and connections for efficient neural network,” ArXiv , vol. abs/1506.02626, 2015
Original
2015
Earlier work this paper cites.
G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” in NIPS Deep Learning and Representation Learning Workshop , 2015. [Online]. Available: http://arxiv.org/abs/1503.02531
Original
2015
Earlier work this paper cites.
S. Srinivas and R. V. Babu, “Data-free parameter pruning for deep neural networks,” in BMVC , 2015
2015
Earlier work this paper cites.
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 , ser. ICML’15. JMLR.org, 2015, p. 448–456
2015
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , Y. Bengio and Y. LeCun, Eds., 2015. [Online]. Available: http://arxiv.org/abs/1412.6980
Original
2015
Earlier work this paper cites.
M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi, “Xnor-net: Imagenet classification using binary convolutional neural networks,” in European Conference on Computer Vision . Springer, 2016, pp. 525–542
2016
Earlier work this paper cites.
H. B. McMahan, E. Moore, D. Ramage, and B. A. y Arcas, “Federated learning of deep networks using model averaging,” CoRR , vol. abs/1602.05629, 2016. [Online]. Available: http://arxiv.org/abs/1602.05629
Original
2016
Earlier work this paper cites.
M. Courbariaux and Y. Bengio, “Binarynet: Training deep neural networks with weights and activations constrained to +1 or -1,” CoRR , vol. abs/1602.02830, 2016. [Online]. Available: http://arxiv.org/abs/1602.02830
Original
2016
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2016
2016
Earlier work this paper cites.
A. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” ArXiv , vol. abs/1704.04861, 2017
Original
2017
Earlier work this paper cites.
J. Redmon and A. Farhadi, “Yolo9000: Better, faster, stronger,” in Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on . IEEE, 2017, pp. 6517–6525
2017
Earlier work this paper cites.
X. Dong, S. Chen, and S. J. Pan, “Learning to prune deep neural networks via layer-wise optimal brain surgeon,” in Proceedings of the 31st International Conference on Neural Information Processing Systems , ser. NIPS’17. Red Hook, NY, USA: Curran Associates Inc., 2017, p. 4860–4874
2017
Earlier work this paper cites.
B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018
2018
Earlier work this paper cites.

Similar

R. Krishnamoorthi, “Quantizing deep convolutional networks for efficient inference: A whitepaper,” CoRR , vol. abs/1806.08342, 2018. [Online]. Available: http://arxiv.org/abs/1806.08342
Original
2018
Cited alongside, same era.
B. Liu, Y. Cao, M. Long, J. Wang, and J. Wang, “Deep triplet quantization,” in Proceedings of the 26th ACM International Conference on Multimedia , ser. MM ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 755–763. [Online]. Available: https://doi.org/10.1145/3240508.3240516
2018
Cited alongside, same era.
R. Arora, A. Basu, P. Mianjy, and A. Mukherjee, “Understanding deep neural networks with rectified linear units,” in International Conference on Learning Representations , 2018. [Online]. Available: https://openreview.net/forum?id=B1J_rgWRW
2018
Cited alongside, same era.
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018

Then

H. Wu, P. Judd, X. Zhang, M. Isaev, and P. Micikevicius, “Integer quantization for deep learning inference: Principles and empirical evaluation,” ArXiv , vol. abs/2004.09602, 2020
Original
2020
Closest in time.
Y. Yang, S. Wu, L. Deng, T. Yan, Y. Xie, and G. Li, “Training high-performance and large-scale deep neural networks with full 8-bit integers,” Neural networks : the official journal of the International Neural Network Society , vol. 125, pp. 70–82, 2020
2020
Closest in time.
G. Di Guglielmo, J. M. Duarte, P. Harris, D. Hoang, S. Jindariani, E. Kreinar, M. Liu, V. Loncar, J. Ngadiuba, K. Pedro, and et al., “Compressing deep neural networks on fpgas to binary and ternary precision with hls4ml,” Machine Learning: Science and Technology , Jun 2020. [Online]. Available: http://dx.doi.org/10.1088/2632-2153/aba042
2020
Closest in time.
B. Martinez, J. Yang, A. Bulat, and G. Tzimiropoulos, “Training binary neural networks with real-to-binary convolutions,” in International Conference on Learning Representations , 2020. [Online]. Available: https://openreview.net/forum?id=BJg4NgBKvH

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2018

Cited alongside, same era.

M. Wortsman, A. Farhadi, and M. Rastegari, “Discovering neural wirings,” in NeurIPS , 2019

2019

Cited alongside, same era.

M. Nagel, M. v. Baalen, T. Blankevoort, and M. Welling, “Data-free quantization through weight equalization and bias correction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , October 2019

2019

Cited alongside, same era.

R. Gong, X. Liu, S. Jiang, T. Li, P. Hu, J. Lin, F. Yu, and J. Yan, “Differentiable soft quantization: Bridging full-precision and low-bit neural networks,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , October 2019

2019

Cited alongside, same era.

R. Li, Y. Wang, F. Liang, H. Qin, J. Yan, and R. Fan, “Fully quantized network for object detection,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2019, pp. 2805–2814

2019

Cited alongside, same era.

A. Bhandare, V. Sripathi, D. Karkada, V. Menon, S. Choi, K. Datta, and V. Saletore, “Efficient 8-bit quantization of transformer neural machine language translation model,” ArXiv , vol. abs/1906.00532, 2019

Original

2019

Cited alongside, same era.

N. Lee, T. Ajanthan, and P. Torr, “SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY,” in International Conference on Learning Representations , 2019. [Online]. Available: https://openreview.net/forum?id=B1VZqjAcYX

2019

Cited alongside, same era.

T. Dettmers and L. Zettlemoyer, “Sparse networks from scratch: Faster training without losing performance,” 2019

2019

Cited alongside, same era.

E. Meller, A. Finkelstein, U. Almog, and M. Grobman, “Same, same but different: Recovering neural network quantization error through weight factorization,” in Proceedings of the 36th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97. PMLR, 09–15 Jun 2019, pp. 4486–4495. [Online]. Available: http://proceedings.mlr.press/v97/meller19a.html

2019

Cited alongside, same era.

2020

Closest in time.

M. Lin, R. Ji, S. Li, Q. Ye, Y. Tian, J. Liu, and Q. Tian, “Filter sketch for network pruning,” ArXiv , vol. abs/2001.08514, 2020

Original

2020

Closest in time.

M. Nagel, R. A. Amjad, M. van Baalen, C. Louizos, and T. Blankevoort, “Up or down? adaptive rounding for post-training quantization,” ArXiv , vol. abs/2004.10568, 2020

Original

2020

Closest in time.

X. Suau, u. Zappella, and N. Apostoloff, “Filter distillation for network compression,” in 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) , 2020, pp. 3129–3138

2020

Closest in time.

H. Yin, P. Molchanov, J. M. Alvarez, Z. Li, A. Mallya, D. Hoiem, N. K. Jha, and J. Kautz, “Dreaming to distill: Data-free knowledge transfer via deepinversion,” in The IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR) , June 2020

2020

Closest in time.

M. Haroush, I. Hubara, E. Hoffer, and D. Soudry, “The knowledge within: Methods for data-free model compression,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020

2020

Closest in time.

Y. Choi, J. Choi, M. El-Khamy, and J. Lee, “Data-free network quantization with adversarial knowledge distillation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , June 2020

2020

Closest in time.

S. Xu, H. Li, B. Zhuang, J. Liu, J. Cao, C. Liang, and M. Tan, “Generative low-bitwidth data free quantization,” in Computer Vision – ECCV 2020 , ser. Lecture Notes in Computer Science, A. Vedaldi, H. Bischof, T. Brox, and J.-M. Frahm, Eds. Springer, 2020, pp. 1–17, european Conference on Computer Vision 2020, ECCV 2020 ; Conference date: 23-08-2020 Through 28-08-2020. [Online]. Available: https://link.springer.com/book/10.1007/978-3-030-58452-8, https://eccv2020.eu

2020

Closest in time.

Y. Cai, Z. Yao, Z. Dong, A. Gholami, M. W. Mahoney, and K. Keutzer, “Zeroq: A novel zero shot quantization framework,” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 13 166–13 175, 2020

2020

Closest in time.

D. Blalock, J. J. G. Ortiz, J. Frankle, and J. Guttag, “What is the state of neural network pruning?” 2020

2020

Closest in time.

Y. Chen, C. Hawkins, K. Zhang, Z. Zhang, and C. Hao, “3u-edgeai: Ultra-low memory training, ultra-low bitwidthquantization, and ultra-low latency acceleration,” CoRR , vol. abs/2105.06250, 2021. [Online]. Available: https://arxiv.org/abs/2105.06250

Original

2021

Closest in time.

J. Tang, M. Liu, N. Jiang, H. Cai, W. Yu, and J. Zhou, “Data-free network pruning for model compression,” in 2021 IEEE International Symposium on Circuits and Systems (ISCAS) , 2021, pp. 1–5

2021

Closest in time.

X. He, Q. Hu, P. Wang, and J. Cheng, “Generative zero-shot network quantization,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , pp. 2994–3005, 2021

2021

Closest in time.

X. Zhang, H. Qin, Y. Ding, R. Gong, Q. Yan, R. Tao, Y. Li, F. Yu, and X. Liu, “Diversifying sample generation for accurate data-free quantization,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 15 653–15 662, 2021

2021

Closest in time.

N. Markus, “Fusing batchnorm with convolution in runtime,” https://nenadmarkus.com/p/fusing-batchnorm-and-conv/, accessed: 2021-12-22

2021

Closest in time.