GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference

A. Fan, E. Grave, and A. Joulin, “Reducing transformer depth on demand with structured dropout,” CoRR , vol. abs/1909.11556, 2019. [Online]. Available: http://arxiv.org/abs/1909.11556

Original

1909

Earlier work this paper cites.

X. Jiao, Y. Yin, L. Shang, X. Jiang, X. Chen, L. Li, F. Wang, and Q. Liu, “Tinybert: Distilling BERT for natural language understanding,” CoRR , vol. abs/1909.10351, 2019. [Online]. Available: http://arxiv.org/abs/1909.10351

Original

1909

Earlier work this paper cites.

Z. Sun, H. Yu, X. Song, R. Liu, Y. Yang, and D. Zhou, “Mobilebert: a compact task-agnostic BERT for resource-limited devices,” CoRR , vol. abs/2004.02984, 2020. [Online]. Available: https://arxiv.org/abs/2004.02984

Original

2004

Earlier work this paper cites.

N. Muralimanohar, R. Balasubramonian, and N. Jouppi, “Cacti 6.0: A tool to model large caches,” HP Laboratories , 01 2009

2009

Earlier work this paper cites.

P. Rosenfeld, E. Cooper-Balis, and B. Jacob, “Dramsim2: A cycle accurate memory system simulator,” IEEE Computer Architecture Letters , vol. 10, no. 1, pp. 16–19, Jan 2011

2011

Earlier work this paper cites.

S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding,” in 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings , Y. Bengio and Y. LeCun, Eds., 2016. [Online]. Available: http://arxiv.org/abs/1510.00149

Original

2016

Earlier work this paper cites.

S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz, and W. J. Dally, “Eie: Efficient inference engine on compressed deep neural network,” in Proceedings of the 43rd International Symposium on Computer Architecture , ser. ISCA ’16. IEEE Press, 2016, p. 243–254. [Online]. Available: https://doi.org/10.1109/ISCA.2016.30

2016

Earlier work this paper cites.

P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “Squad: 100,000+ questions for machine comprehension of text,” arXiv preprint arXiv:1606.05250 , 2016

Original

2016

Earlier work this paper cites.

Y.-H. Chen, J. Emer, and V. Sze, “Eyeriss: A spatial architecture for energy-efficient dataflow for convolutional neural networks,” in Intl’ Symp. on Computer Architecture , 2016

2016

Earlier work this paper cites.

Y. Cheng, D. Wang, P. Zhou, and T. Zhang, “A survey of model compression and acceleration for deep neural networks,” 2017

2017

Earlier work this paper cites.

N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borchers, R. Boyle, P.-l. Cantin, C. Chao, C. Clark, J. Coriell, M. Daley, M. Dau, J. Dean, B. Gelb, T. V. Ghaemmaghami, R. Gottipati, W. Gulland, R. Hagmann, C. R. Ho, D. Hogberg, J. Hu, R. Hundt, D. Hurt, J. Ibarz, A. Jaffey, A. Jaworski, A. Kaplan, H. Khaitan, D. Killebrew, A. Koch, N. Kumar, S. Lacy, J. Laudon, J. Law, D. Le, C. Leary, Z. Liu, K. Lucke, A. Lundin, G. MacKean, A. Maggiore, M. Mahony, K. Miller, R. Nagarajan, R. Narayanaswami, R. Ni, K. Nix, T. Norrie, M. Omernick, N. Penukonda, A. Phelps, J. Ross, M. Ross, A. Salek, E. Samadiani, C. Severn, G. Sizikov, M. Snelham, J. Souter, D. Steinberg, A. Swing, M. Tan, G. Thorson, B. Tian, H. Toma, E. Tuttle, V. Vasudevan, R. Walter, W. Wang, E. Wilcox, and D. H. Yoon, “In-datacenter performance analysis of a tensor processing unit,” in International Symposium on Computer Architecture , 2017

2017

Earlier work this paper cites.

L. Durant, O. Giroux, M. Harris, and N. Stam, “Nvidia developer blog,” May 2017. [Online]. Available: https://devblogs.nvidia.com/inside-volta/

2017

Earlier work this paper cites.

E. Park, S. Yoo, and P. Vajda, “Value-aware quantization for training and inference of neural networks,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 580–595

2018

Earlier work this paper cites.

E. Park, D. Kim, and S. Yoo, Energy-Efficient Neural Network Accelerator Based on Outlier-Aware Low-Precision Computation . IEEE Press, 2018, p. 688–698. [Online]. Available: https://doi.org/10.1109/ISCA.2018.00063

2018

Cited alongside, same era.

Z. Jia, M. Maggioni, B. Staiger, and D. P. Scarpazza, “Dissecting the NVIDIA volta GPU architecture via microbenchmarking,” CoRR , vol. abs/1804.06826, 2018. [Online]. Available: http://arxiv.org/abs/1804.06826

Original

2018

Cited alongside, same era.

B. Fleischer, S. Shukla, M. Ziegler, J. Silberman, J. Oh, V. Srinivasan, J. Choi, S. Mueller, A. Agrawal, T. Babinsky, N. Cao, C. Chen, P. Chuang, T. Fox, G. Gristede, M. Guillorn, H. Haynie, M. Klaiber, D. Lee, S. Lo, G. Maier, M. Scheuermann, S. Venkataramani, C. Vezyrtzis, N. Wang, F. Yee, C. Zhou, P. Lu, B. Curran, L. Chang, and K. Gopalakrishnan, “A scalable multi- teraops deep learning processor core for ai training and inference,” in 2018 IEEE Symposium on VLSI Circuits , 2018, pp. 35–36

Original

2018

Cited alongside, same era.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in NAACL-HLT , 2019

S. Sun, Y. Cheng, Z. Gan, and J. Liu, “Patient knowledge distillation for bert model compression,” arXiv preprint arXiv:1908.09355 , 2019

Original

2019

Later among the works it cites.

R. Tang, Y. Lu, L. Liu, L. Mou, O. Vechtomova, and J. Lin, “Distilling task-specific knowledge from bert into simple neural networks,” arXiv preprint arXiv:1903.12136 , 2019

Original

2019

Later among the works it cites.

R. Tang, Y. Lu, and J. Lin, “Natural language generation for effective knowledge distillation,” in Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019) , 2019, pp. 202–208

2019

Later among the works it cites.

Y. K. Chia, S. Witteveen, and M. Andrews, “Transformer to cnn: Label-scarce distillation for efficient text classification,” arXiv preprint arXiv:1909.03508 , 2019

Original

2019

Later among the works it cites.