Dithered backprop: A sparse and quantized backpropagation algorithm for more efficient deep neural network training

Learning representations by back-propagating errors

David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams · 1986

Earlier work this paper cites.

Online learning and stochastic approximations, 1998

Léon Bottou · 1998

Earlier work this paper cites.

Training deep neural networks with low precision multiplications, 2014

Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David · 2014

Earlier work this paper cites.

Deep learning with limited numerical precision

Suyog Gupta, Ankur Agrawal, Kailash Gopalakrishnan, and Pritish Narayanan · 2015

Earlier work this paper cites.

Deep Learning

Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio · 2016

Earlier work this paper cites.

Binarized neural networks: Training deep neural networks with weights and activations constrained to +1 or -1, 2016

Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio · 2016

Earlier work this paper cites.

Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients, 2016

Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou · 2016

Earlier work this paper cites.

Eie: Efficient inference engine on compressed deep neural network

S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz, and W. J. Dally · 2016

Earlier work this paper cites.

Efficient processing of deep neural networks: A tutorial and survey

V. Sze, Y. Chen, T. Yang, and J. S. Emer · 2017

Earlier work this paper cites.

Quantized neural networks: Training neural networks with low precision weights and activations

Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio · 2017

Earlier work this paper cites.

Terngrad: Ternary gradients to reduce communication in distributed deep learning

Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li · 2017

Cited alongside, same era.

meprop: Sparsified back propagation for accelerated deep learning with reduced overfitting, 2017

Xu Sun, Xuancheng Ren, Shuming Ma, and Houfeng Wang · 2017

Cited alongside, same era.

Scnn: An accelerator for compressed-sparse convolutional neural networks

A. Parashar, M. Rhu, A. Mukkara, A. Puglielli, R. Venkatesan, B. Khailany, J. Emer, S. W. Keckler, and W. J. Dally · 2017

Cited alongside, same era.

Model compression and acceleration for deep neural networks: The principles, progress, and challenges

Y. Cheng, D. Wang, P. Zhou, and T. Zhang · 2018

Cited alongside, same era.

Mixed precision training

Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu · 2018

Cited alongside, same era.

Energy and policy considerations for deep learning in NLP

Original

Emma Strubell, Ananya Ganesh, and Andrew McCallum · 2019

Later among the works it cites.

Entropy-constrained training of deep neural networks

S. Wiedemann, A. Marban, K. Müller, and W. Samek · 2019

Later among the works it cites.

Fast sparse convnets, 2019

Erich Elsen, Marat Dukhan, Trevor Gale, and Karen Simonyan · 2019

Later among the works it cites.

Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices

Y. Chen, T. Yang, J. Emer, and V. Sze · 2019

Later among the works it cites.

Robust and communication-efficient federated learning from non-i.i.d. data

F. Sattler, S. Wiedemann, K. Müller, and W. Samek · 2019

Later among the works it cites.

Sparse binary compression: Towards distributed deep learning with minimal communication