NVIDIA Tensor Core Programmability, Performance & Precision

N. J. Higham, “The accuracy of floating point summation,” SIAM Journal on Scientific Computing , vol. 14, no. 4, pp. 783–799, 1993

1993

Earlier work this paper cites.

A. Buttari, J. Dongarra, J. Langou, J. Langou, P. Luszczek, and J. Kurzak, “Mixed precision iterative refinement techniques for the solution of dense linear systems,” The International Journal of High Performance Computing Applications , vol. 21, no. 4, pp. 457–466, 2007

2007

Earlier work this paper cites.

M. M. Khan, D. R. Lester, L. A. Plana, A. Rast, X. Jin, E. Painkras, and S. B. Furber, “SpiNNaker: mapping neural networks onto a massively-parallel chip multiprocessor,” in Neural Networks, 2008. , 2008, pp. 2849–2856

2008

Earlier work this paper cites.

NVIDIA, “cuBLAS library,” NVIDIA Corporation, Santa Clara, California , vol. 15, no. 27, p. 31, 2008

2008

Earlier work this paper cites.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Advances in neural information processing systems , 2012, pp. 1097–1105

2012

Earlier work this paper cites.

A. Putnam, A. M. Caulfield, E. S. Chung, D. Chiou, K. Constantinides, J. Demme, H. Esmaeilzadeh, J. Fowers, G. P. Gopal, J. Gray et al. , “A reconfigurable fabric for accelerating large-scale datacenter services,” in Computer Architecture (ISCA) . IEEE, 2014, pp. 13–24

2014

Earlier work this paper cites.

P. A. Merolla, J. V. Arthur, R. Alvarez-Icaza, A. S. Cassidy, J. Sawada, F. Akopyan, B. L. Jackson, N. Imam, C. Guo, Y. Nakamura et al. , “A million spiking-neuron integrated circuit with a scalable communication network and interface,” Science , vol. 345, no. 6197, pp. 668–673, 2014

2014

Earlier work this paper cites.

M. Courbariaux, J.-P. David, and Y. Bengio, “Low precision storage for deep learning,” arXiv preprint Arxiv:1412.7024 , 2014

Original

2014

Earlier work this paper cites.

S. Chetlur, C. Woolley, P. Vandermersch, J. Cohen, J. Tran, B. Catanzaro, and E. Shelhamer, “cuDNN: Efficient primitives for deep learning,” arXiv preprint arXiv:1410.0759 , 2014

Original

2014

Earlier work this paper cites.

B. Barry, C. Brick, F. Connor, D. Donohoe, D. Moloney, R. Richmond, M. O’Riordan, and V. Toma, “Always-on vision processing unit for mobile applications,” IEEE Micro , vol. 35, no. 2, pp. 56–66, 2015

2015

Earlier work this paper cites.

S. Gupta, A. Agrawal, K. Gopalakrishnan, and P. Narayanan, “Deep learning with limited numerical precision,” in International Conference on Machine Learning , 2015, pp. 1737–1746

2015

Cited alongside, same era.

S. Markidis, J. Gong, M. Schliephake, E. Laure, A. Hart, D. Henty, K. Heisey, and P. Fischer, “OpenACC acceleration of the Nek5000 spectral element code,” The International Journal of High Performance Computing Applications , vol. 29, no. 3, pp. 311–319, 2015

2015

Cited alongside, same era.

I. Goodfellow, Y. Bengio, and A. Courville, Deep learning . MIT press, 2016

2016

Cited alongside, same era.

M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard et al. , “TensorFlow: A system for large-scale machine learning.” in OSDI , vol. 16, 2016, pp. 265–283

2016

Cited alongside, same era.

U. Köster, T. Webb, X. Wang, M. Nassar, A. K. Bansal, W. Constable, O. Elibol, S. Hall, L. Hornof, A. Khosrowshahi et al. , “Flexpoint: An adaptive numerical format for efficient training of deep neural networks,” in Advances in Neural Information Processing Systems , 2017, pp. 1740–1750

2017

Later among the works it cites.

P. Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaev, G. Venkatesh et al. , “Mixed precision training,” arXiv preprint arXiv:1710.03740 , 2017

Original

2017

Later among the works it cites.

A. Haidar, P. Wu, S. Tomov, and J. Dongarra, “Investigating half precision arithmetic to accelerate dense linear system solvers,” in Proceedings of the 8th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems . ACM, 2017, p. 10

2017

Later among the works it cites.

L. Durant, O. Giroux, M. Harris, and N. Stam, “Inside Volta: The world’s most advanced data center GPU,” 2017, accessed: 2017-01-27. [Online]. Available: https://devblogs.nvidia.com/inside-volta/