Survey and Benchmarking of Machine Learning Accelerators

M. L. Minsky, Computation: Finite and Infinite Machines . Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 1967

1967

Earlier work this paper cites.

S. Williams, A. Waterman, and D. Patterson, “Roofline: An Insightful Visual Performance Model for Multicore Architectures,” Commun. ACM , vol. 52, no. 4, pp. 65–76, apr 2009. [Online]. Available: http://doi.acm.org/10.1145/1498765.1498785

2009

Earlier work this paper cites.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” Neural Information Processing Systems , vol. 25, 2012

2012

Earlier work this paper cites.

M. Horowitz, “Computing’s Energy Problem (and What We Can Do About It),” in 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC) . IEEE, feb 2014, pp. 10–14. [Online]. Available: http://ieeexplore.ieee.org/document/6757323/

2014

Earlier work this paper cites.

Y. Chen, T. Luo, S. Liu, S. Zhang, L. He, J. Wang, L. Li, T. Chen, Z. Xu, N. Sun, and O. Temam, “DaDianNao: A Machine-Learning Supercomputer,” in 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture . IEEE, dec 2014, pp. 609–622. [Online]. Available: http://ieeexplore.ieee.org/document/7011421/

2014

Earlier work this paper cites.

R. Smith, “NVIDIA Launches Tesla K80, GK210 GPU,” nov 2014. [Online]. Available: https://www.anandtech.com/show/8729/nvidia-launches-tesla-k80-gk210-gpu

2014

Earlier work this paper cites.

T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft COCO: Common Objects in Context,” in Computer Vision – ECCV 2014 . Springer International Publishing, 2014, pp. 740–755. [Online]. Available: http://link.springer.com/10.1007/978-3-319-10602-1_48

2014

Earlier work this paper cites.

S. Gupta, A. Agrawal, K. Gopalakrishnan, and P. Narayanan, “Deep Learning with Limited Numerical Precision,” in Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37 , ser. ICML’15. JMLR.org, 2015, pp. 1737–1746. [Online]. Available: http://dl.acm.org/citation.cfm?id=3045118.3045303

2015

Earlier work this paper cites.

F. Akopyan, J. Sawada, A. Cassidy, R. Alvarez-Icaza, J. Arthur, P. Merolla, N. Imam, Y. Nakamura, P. Datta, G. Nam, B. Taba, M. Beakes, B. Brezzo, J. B. Kuang, R. Manohar, W. P. Risk, B. Jackson, and D. S. Modha, “TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 34, no. 10, pp. 1537–1557, oct 2015

2015

Earlier work this paper cites.

Z. Du, R. Fasthuber, T. Chen, P. Ienne, L. Li, T. Luo, X. Feng, Y. Chen, and O. Temam, “ShiDianNao: Shifting vision processing closer to the sensor,” in ACM SIGARCH Computer Architecture News , vol. 43, no. 3. ACM, 2015, pp. 92–104

2015

Earlier work this paper cites.

D. Liu, T. Chen, S. Liu, J. Zhou, S. Zhou, O. Teman, X. Feng, X. Zhou, and Y. Chen, “Pudiannao: A polyvalent machine learning accelerator,” in ACM SIGARCH Computer Architecture News , vol. 43, no. 1. ACM, 2015, pp. 369–381

2015

Earlier work this paper cites.

A. Canziani, A. Paszke, and E. Culurciello, “An Analysis of Deep Neural Network Models for Practical Applications,” arXiv preprint arXiv:1605.07678 , 2016

Original

2016

Earlier work this paper cites.

M. Feldman, “IBM Finds Killer App for TrueNorth Neuromorphic Chip,” sep 2016. [Online]. Available: https://www.top500.org/news/ibm-finds-killer-app-for-truenorth-neuromorphic-chip/

2016

Earlier work this paper cites.

Y. Chen, T. Chen, Z. Xu, N. Sun, and O. Temam, “DianNao Family: Energy-Efficient Accelerators For Machine Learning,” Communications of the ACM , vol. 59, no. 11, pp. 105–112, oct 2016. [Online]. Available: http://dl.acm.org/citation.cfm?doid=3013530.2996864

2016

Earlier work this paper cites.

C. Zhang, D. Wu, J. Sun, G. Sun, G. Luo, and J. Cong, “Energy-Efficient CNN Implementation on a Deeply Pipelined FPGA Cluster,” in Proceedings of the 2016 International Symposium on Low Power Electronics and Design , ser. ISLPED ’16. New York, NY, USA: ACM, 2016, pp. 326–331. [Online]. Available: http://doi.acm.org/10.1145/2934583.2934644

2016

Earlier work this paper cites.

J. Jeffers, J. Reinders, and A. Sodani, Intel Xeon Phi Processor High Performance Programming: Knights Landing Edition 2Nd Edition , 2nd ed. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2016

2016

Earlier work this paper cites.

R. Smith, “NVIDIA Announces Tesla P100 Accelerator - Pascal GP100 Power for HPC,” apr 2016. [Online]. Available: https://www.anandtech.com/show/10222/nvidia-announces-tesla-p100-accelerator-pascal-power-for-hpc

2016

Earlier work this paper cites.

A. Olofsson, “Epiphany-V: A 1024-core 64-bit RISC processor — Parallella,” oct 2016. [Online]. Available: https://www.parallella.org/2016/10/05/epiphany-v-a-1024-core-64-bit-risc-processor/

2016

Earlier work this paper cites.

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: Single Shot MultiBox Detector,” in Computer Vision – ECCV 2016 . Springer International Publishing, 2016, pp. 21–37. [Online]. Available: http://link.springer.com/10.1007/978-3-319-46448-0_2

2016

Earlier work this paper cites.

T. N. Theis and H. . P. Wong, “The End of Moore’s Law: A New Beginning for Information Technology,” Computing in Science Engineering , vol. 19, no. 2, pp. 41–50, mar 2017

2017

Earlier work this paper cites.

V. Sze, Y. Chen, T. Yang, and J. S. Emer, “Efficient Processing of Deep Neural Networks: A Tutorial and Survey,” Proceedings of the IEEE , vol. 105, no. 12, pp. 2295–2329, dec 2017

2017

Earlier work this paper cites.

Y. Chen, T. Krishna, J. S. Emer, and V. Sze, “Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks,” IEEE Journal of Solid-State Circuits , vol. 52, no. 1, pp. 127–138, jan 2017

2017

Earlier work this paper cites.

J. Hruska, “New Movidius Myriad X VPU Packs a Custom Neural Compute Engine,” aug 2017

2017

Earlier work this paper cites.

D. Franklin, “NVIDIA Jetson TX2 Delivers Twice the Intelligence to the Edge,” mar 2017

2017

Earlier work this paper cites.

Z. Li, Y. Wang, T. Zhi, and T. Chen, “A survey of neural network accelerators,” Frontiers of Computer Science , vol. 11, no. 5, pp. 746–761, oct 2017. [Online]. Available: http://link.springer.com/10.1007/s11704-016-6159-1

2017

Earlier work this paper cites.

K. Guo, S. Zeng, J. Yu, Y. Wang, and H. Yang, “A Survey of FPGA-Based Neural Network Accelerator,” arXiv preprint arXiv:1712.08934 , dec 2017. [Online]. Available: http://arxiv.org/abs/1712.08934

Original

2017

Earlier work this paper cites.

H. Nakahara, T. Fujii, and S. Sato, “A Fully Connected Layer Elimination for a Binarizec Convolutional Neural Network on an FPGA,” in 2017 27th International Conference on Field Programmable Logic and Applications (FPL) , 2017, pp. 1–4

2017

Earlier work this paper cites.

L. Jiao, C. Luo, W. Cao, X. Zhou, and L. Wang, “Accelerating Low Bit-Width Convolutional Neural Networks with Embedded FPGA,” in 2017 27th International Conference on Field Programmable Logic and Applications (FPL) . IEEE, sep 2017, pp. 1–4. [Online]. Available: http://ieeexplore.ieee.org/document/8056820/

2017

Earlier work this paper cites.

S. Han, J. Kang, H. Mao, Y. Hu, X. Li, Y. Li, D. Xie, H. Luo, S. Yao, Y. Wang, H. Yang, and W. B. J. Dally, “ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA,” in Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays , ser. FPGA ’17. New York, NY, USA: ACM, 2017, pp. 75–84. [Online]. Available: http://doi.acm.org/10.1145/3020078.3021745

2017

Earlier work this paper cites.

L. Lu, Y. Liang, Q. Xiao, and S. Yan, “Evaluating Fast Algorithms for Convolutional Neural Networks on FPGAs,” in 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) . IEEE, apr 2017, pp. 101–108. [Online]. Available: http://ieeexplore.ieee.org/document/7966660/

2017

Earlier work this paper cites.

A. Podili, C. Zhang, and V. Prasanna, “Fast and efficient implementation of Convolutional Neural Networks on FPGA,” in 2017 IEEE 28th International Conference on Application-specific Systems, Architectures and Processors (ASAP) . IEEE, jul 2017, pp. 11–18. [Online]. Available: http://ieeexplore.ieee.org/document/7995253/

2017

Earlier work this paper cites.

Y. Ma, Y. Cao, S. Vrudhula, and J.-s. Seo, “Optimizing Loop Operation and Dataflow in FPGA Acceleration of Deep Convolutional Neural Networks,” in Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays , ser. FPGA ’17. New York, NY, USA: ACM, 2017, pp. 45–54. [Online]. Available: http://dl.acm.org/citation.cfm?doid=3020078.3021736 http://doi.acm.org/10.1145/3020078.3021736

2017

Cited alongside, same era.

J. Zhang and J. Li, “Improving the Performance of OpenCL-based FPGA Accelerator for Convolutional Neural Network,” in Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays , ser. FPGA ’17. New York, NY, USA: ACM, 2017, pp. 25–34. [Online]. Available: http://doi.acm.org/10.1145/3020078.3021698

2017

Cited alongside, same era.

U. Aydonat, S. O’Connell, D. Capalija, A. C. Ling, and G. R. Chiu, “An OpenCL \ \backslash texttrademark Deep Learning Accelerator on Arria 10,” in Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays , ser. FPGA ’17. New York, NY, USA: ACM, 2017, pp. 55–64. [Online]. Available: http://dl.acm.org/citation.cfm?doid=3020078.3021738 http://doi.acm.org/10.1145/3020078.3021738

L. Armasu, “Move Over GPUs: Startup’s Chip Claims to Do Deep Learning Inference Better,” sep 2018. [Online]. Available: https://www.tomshardware.com/news/habana-inference-goya-custom-chip,37821.html

2018

Later among the works it cites.

I. Cutress, “Cambricon, Maker of Hauwei’s Kirin NPU IP, Build a Big AI Chip and PCIe Card,” may 2018. [Online]. Available: https://www.anandtech.com/show/12815/cambricon-makers-of-huaweis-kirin-npu-ip-build-a-big-ai-chip-and-pcie-card

2018

Later among the works it cites.

R. Merritt, “Baidu Accelerator Rises in AI,” jul 2018. [Online]. Available: https://www.eetimes.com/document.asp?doc_id=1333449

2018

Later among the works it cites.

C. Duckett, “Baidu Creates Kunlun Silicon for AI,” jul 2018. [Online]. Available: https://www.zdnet.com/article/baidu-creates-kunlun-silicon-for-ai/

2018

Later among the works it cites.