Built on

S. Hooker, A. Courville, G. Clark, Y. Dauphin, and A. Frome, “What do compressed deep neural networks forget?” arXiv preprint arXiv:1911.05248 , 2021. [Online]. Available: https://arxiv.org/abs/1911.05248
Original
1911
Earlier work this paper cites.
A. Oppenheim, “Realization of digital filters using block-floating-point arithmetic,” IEEE transactions on audio and electroacoustics , vol. 18, no. 2, pp. 130–136, 1970
1970
Earlier work this paper cites.
Y. LeCun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” in Advances in neural information processing systems , 1990, pp. 598–605
1990
Earlier work this paper cites.
H. Wu, P. Judd, X. Zhang, M. Isaev, and P. Micikevicius, “Integer quantization for deep learning inference: Principles and empirical evaluation,” CoRR , vol. abs/2004.09602, 2020. [Online]. Available: https://arxiv.org/abs/2004.09602
Original
2004
Earlier work this paper cites.
C. Bucilua, R. Caruana, and A. Niculescu-Mizil, “Model compression,” in Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining , 2006, pp. 535–541
2006
Earlier work this paper cites.
K. Chellapilla, S. Puri, and P. Simard, “High performance convolutional neural networks for document processing,” in Tenth international workshop on frontiers in handwriting recognition . Suvisoft, 2006
2006
Earlier work this paper cites.
C.-C. Huang, S.-H. Hung, J.-F. Chung, L.-D. Van, and C.-T. Lin, “Front-end amplifier of low-noise and tunable bw/gain for portable biomedical signal acquisition,” in 2008 IEEE International Symposium on Circuits and Systems . IEEE, 2008, pp. 2717–2720
2008
Earlier work this paper cites.
A. Dasgupta, R. Kumar, and T. Sarlós, “A sparse johnson: Lindenstrauss transform,” in Proceedings of the forty-second ACM symposium on Theory of computing , 2010, pp. 341–350
2010
Earlier work this paper cites.
E. Liberty, “Simple and deterministic matrix sketching,” in Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining , 2013, pp. 581–588
2013
Earlier work this paper cites.
Y. Bengio, N. Léonard, and A. Courville, “Estimating or propagating gradients through stochastic neurons for conditional computation,” arXiv preprint arXiv:1308.3432 , 2013
Original
2013
Earlier work this paper cites.
W. Dally, “High-performance hardware for machine learning,” NIPS Tutorial , vol. 2, 2015
2015
Earlier work this paper cites.
G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015
Original
2015
Earlier work this paper cites.
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin et al. , “Tensorflow: Large-scale machine learning on heterogeneous distributed systems,” arXiv preprint arXiv:1603.04467 , 2016
Original
2016
Earlier work this paper cites.
T. NVIDIA, “V100 gpu architecture,” The world’s most advanced data center GPU. Version WP-08608-001_v1 , vol. 1, 2017
2017
Earlier work this paper cites.
N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borchers et al. , “In-datacenter performance analysis of a tensor processing unit,” in Proceedings of the 44th annual international symposium on computer architecture , 2017, pp. 1–12
2017
Earlier work this paper cites.

Similar

Then

J. L. McKinstry, S. K. Esser, R. Appuswamy, D. Bablani, J. V. Arthur, I. B. Yildiz, and D. S. Modha, “Discovering low-precision networks close to full-precision networks for efficient inference,” in 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition (EMC2-NIPS) , 2019, pp. 6–9
2019
Later among the works it cites.
M. Nagel, M. V. Baalen, T. Blankevoort, and M. Welling, “Data-free quantization through weight equalization and bias correction,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV) , 2019, pp. 1325–1334
2019
Later among the works it cites.
L. Deng, G. Li, S. Han, L. Shi, and Y. Xie, “Model compression and hardware acceleration for neural networks: A comprehensive survey,” Proceedings of the IEEE , vol. 108, no. 4, pp. 485–532, 2020
2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2022

Adaptive Block Floating-Point for Analog Deep Learning Hardware

Basumallik, Ayon, Bunandar, Darius, Dronen, Nicholas et al.

Understand

Analog mixed-signal (AMS) devices promise faster, more energy-efficient deep neural network (DNN) inference than their digital counterparts.

However, recent studies show that DNNs on AMS devices with fixed-point numbers can incur an accuracy penalty because of precision loss.
To mitigate this penalty, we present a novel AMS-compatible adaptive block floating-point (ABFP) number representation.
We also introduce amplification (or gain) as a method for increasing the accuracy of the number representation without increasing the bit precision of the output.

Built on

S. Hooker, A. Courville, G. Clark, Y. Dauphin, and A. Frome, “What do compressed deep neural networks forget?” arXiv preprint arXiv:1911.05248 , 2021. [Online]. Available: https://arxiv.org/abs/1911.05248
Original
1911
Earlier work this paper cites.
A. Oppenheim, “Realization of digital filters using block-floating-point arithmetic,” IEEE transactions on audio and electroacoustics , vol. 18, no. 2, pp. 130–136, 1970
1970
Earlier work this paper cites.
Y. LeCun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” in Advances in neural information processing systems , 1990, pp. 598–605
1990
Earlier work this paper cites.
H. Wu, P. Judd, X. Zhang, M. Isaev, and P. Micikevicius, “Integer quantization for deep learning inference: Principles and empirical evaluation,” CoRR , vol. abs/2004.09602, 2020. [Online]. Available: https://arxiv.org/abs/2004.09602
Original
2004
Earlier work this paper cites.
C. Bucilua, R. Caruana, and A. Niculescu-Mizil, “Model compression,” in Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining , 2006, pp. 535–541
2006
Earlier work this paper cites.
K. Chellapilla, S. Puri, and P. Simard, “High performance convolutional neural networks for document processing,” in Tenth international workshop on frontiers in handwriting recognition . Suvisoft, 2006
2006
Earlier work this paper cites.
C.-C. Huang, S.-H. Hung, J.-F. Chung, L.-D. Van, and C.-T. Lin, “Front-end amplifier of low-noise and tunable bw/gain for portable biomedical signal acquisition,” in 2008 IEEE International Symposium on Circuits and Systems . IEEE, 2008, pp. 2717–2720
2008
Earlier work this paper cites.
A. Dasgupta, R. Kumar, and T. Sarlós, “A sparse johnson: Lindenstrauss transform,” in Proceedings of the forty-second ACM symposium on Theory of computing , 2010, pp. 341–350
2010
Earlier work this paper cites.
E. Liberty, “Simple and deterministic matrix sketching,” in Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining , 2013, pp. 581–588
2013
Earlier work this paper cites.
Y. Bengio, N. Léonard, and A. Courville, “Estimating or propagating gradients through stochastic neurons for conditional computation,” arXiv preprint arXiv:1308.3432 , 2013
Original
2013
Earlier work this paper cites.
W. Dally, “High-performance hardware for machine learning,” NIPS Tutorial , vol. 2, 2015
2015
Earlier work this paper cites.
G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015
Original
2015
Earlier work this paper cites.
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin et al. , “Tensorflow: Large-scale machine learning on heterogeneous distributed systems,” arXiv preprint arXiv:1603.04467 , 2016
Original
2016
Earlier work this paper cites.
T. NVIDIA, “V100 gpu architecture,” The world’s most advanced data center GPU. Version WP-08608-001_v1 , vol. 1, 2017
2017
Earlier work this paper cites.
N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borchers et al. , “In-datacenter performance analysis of a tensor processing unit,” in Proceedings of the 44th annual international symposium on computer architecture , 2017, pp. 1–12
2017
Earlier work this paper cites.

Similar

Then

J. L. McKinstry, S. K. Esser, R. Appuswamy, D. Bablani, J. V. Arthur, I. B. Yildiz, and D. S. Modha, “Discovering low-precision networks close to full-precision networks for efficient inference,” in 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition (EMC2-NIPS) , 2019, pp. 6–9
2019
Later among the works it cites.
M. Nagel, M. V. Baalen, T. Blankevoort, and M. Welling, “Data-free quantization through weight equalization and bias correction,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV) , 2019, pp. 1325–1334
2019
Later among the works it cites.
L. Deng, G. Li, S. Han, L. Shi, and Y. Xie, “Model compression and hardware acceleration for neural networks: A comprehensive survey,” Proceedings of the IEEE , vol. 108, no. 4, pp. 485–532, 2020
2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2018

Cited alongside, same era.

D. S. Khudia, P. Basu, and S. Deng, “Open-sourcing fbgemm for state-of-the-art server-side inference,” 2018. [Online]. Available: https://engineering.fb.com/2018/11/07/ml-applications/fbgemm/

2018

Cited alongside, same era.

D. Fick and M. Henry, “Analog computation in flash memory for datacenter-scale ai inference in a small chip,” Hot Chips 2018 , 2018

2018

Cited alongside, same era.

Z. Song, Z. Liu, and D. Wang, “Computation error analysis of block floating point arithmetic oriented convolution neural network accelerator design,” in Thirty-Second AAAI Conference on Artificial Intelligence , 2018

2018

Cited alongside, same era.

M. Drumond, T. Lin, M. Jaggi, and B. Falsafi, “Training dnns with hybrid block floating point,” Advances in Neural Information Processing Systems , vol. 31, pp. 453–463, 2018

2018

Cited alongside, same era.

B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 2704–2713

2018

Cited alongside, same era.

R. Krishnamoorthi, “Quantizing deep convolutional networks for efficient inference: A whitepaper,” arXiv preprint arXiv:1806.08342 , 2018

Original

2018

Cited alongside, same era.

M. Seok, M. Yang, Z. Jiang, A. A. Lazar, and J.-S. Seo, “Cases for analog mixed signal computing integrated circuits for deep neural networks,” in 2019 International Symposium on VLSI Design, Automation and Test (VLSI-DAT) . IEEE, 2019, pp. 1–2

2019

Cited alongside, same era.

A. S. Rekhi, B. Zimmer, N. Nedovic, N. Liu, R. Venkatesan, M. Wang, B. Khailany, W. J. Dally, and C. T. Gray, “Analog/mixed-signal hardware error modeling for deep learning inference,” in Proceedings of the 56th Annual Design Automation Conference 2019 , ser. DAC ’19. New York, NY, USA: Association for Computing Machinery, 2019. [Online]. Available: https://doi.org/10.1145/3316781.3317770

2019

Cited alongside, same era.

S. Ghodrati, H. Sharma, S. Kinzer, A. Yazdanbakhsh, J. Park, N. S. Kim, D. Burger, and H. Esmaeilzadeh, “Mixed-signal charge-domain acceleration of deep neural networks through interleaved bit-partitioned arithmetic,” in Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques , ser. PACT ’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 399–411. [Online]. Available: https://doi.org/10.1145/3410463.3414634

2020