Fetching the paper…
Reading the bibliography…
Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product operations to preserve the quality of convergence.
Best “ordering” for floating-point addition
Robertazzi, T. G. and Schwartz, S. C. (1988) · 1988
Earlier work this paper cites.
The accuracy of floating point summation
Higham, N. J. (1993) · 1993
Earlier work this paper cites.
Reducing floating point error in dot product using the superblock family of algorithms
Castaldo, A. M., Whaley, R. C., and Chronopoulos, A. T. (2008) · 2008
Earlier work this paper cites.
Quantization noise
Widrow, B. and Kollár, I. (2008) · 2008
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Glorot, X. and Bengio, Y. (2010) · 2010
Earlier work this paper cites.
Deep learning with limited numerical precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., and Narayanan, P. (2015) · 2015
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
He, K., Zhang, X., Ren, S., and Sun, J. (2015) · 2015
Earlier work this paper cites.
Binarized neural networks
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., and Bengio, Y. (2016) · 2016
Cited alongside, same era.
Fixed point quantization of deep convolutional networks
Lin, D., Talathi, S., and Annapureddy, S. (2016) · 2016
Cited alongside, same era.
Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients
Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., and Zou, Y. (2016) · 2016
Cited alongside, same era.
Zhu, C., Han, S., Mao, H., and Dally, W. J. (2016) · 2016
Cited alongside, same era.
Gastaldi, X. (2017) · 2017
Cited alongside, same era.
Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaev, O., Venkatesh, G., et al. (2017) · 2017
Later among the works it cites.
Analytical guarantees on numerical precision of deep neural networks
Sakr, C., Kim, Y., and Shanbhag, N. (2017) · 2017
Later among the works it cites.
Exploiting approximate computing for deep learning acceleration
Chen, C.-Y., Choi, J., Gopalakrishnan, K., Srinivasan, V., and Venkataramani, S. (2018) · 2018
Later among the works it cites.
Discovering low-precision networks close to full-precision networks for efficient embedded inference
McKinstry, J. L., Esser, S. K., Appuswamy, R., Bablani, D., Arthur, J. V., Yildiz, I. B., and Modha, D. S. (2018) · 2018
Later among the works it cites.
Training deep neural networks with 8-bit floating point numbers
Wang, N., Choi, J., Brand, D., Chen, C.-Y., and Gopalakrishnan, K. (2018) · 2018
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., and He, K. (2017) · 2017
Cited alongside, same era.
Flexpoint: An adaptive numerical format for efficient training of deep neural networks
Köster, U., Webb, T., Wang, X., Nassar, M., Bansal, A. K., Constable, W., Elibol, O., Gray, S., Hall, S., Hornof, L., et al. (2017) · 2017
Cited alongside, same era.
Han, S., Mao, H., and Dally, W. J. (2015a)
Cited in the paper.
Learning both weights and connections for efficient neural network
Han, S., Pool, J., Tran, J., and Dally, W. (2015b)
Cited in the paper.
Later among the works it cites.
Training and inference with integers in deep neural networks
Wu, S., Li, G., Chen, F., and Shi, L. (2018) · 2018
Later among the works it cites.