2017

NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps

Aimar, Alessandro, Mostafa, Hesham, Calabrese, Enrico et al.

Understand

Convolutional neural networks (CNNs) have become the dominant neural network architecture for solving many state-of-the-art (SOA) visual processing tasks.

Even though Graphical Processing Units (GPUs) are most often used in training and deploying CNNs, their power efficiency is less than 10 GOp/s/W for single-frame runtime inference.
We propose a flexible and efficient CNN accelerator architecture called NullHop that implements SOA CNNs useful for low-power and low-latency application scenarios.
NullHop exploits the sparsity of neuron activations in CNNs to accelerate the computation and reduce memory requirements.

Built on

R. H. Hahnloser, R. Sarpeshkar, M. A. Mahowald, R. J. Douglas, and H. S. Seung, “Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit,” Nature , vol. 405, no. 6789, pp. 947–951, Jun. 2000. [Online]. Available: http://dx.doi.org/10.1038/35016072
2000
Earlier work this paper cites.
V. Nair and G. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th International Conference on Machine Learning (ICML-10) , 2010, pp. 807–814
2010
Earlier work this paper cites.
D. Scherer, A. Müller, and S. Behnke, “Evaluation of pooling operations in convolutional architectures for object recognition,” in International Conference on Artificial Neural Networks . Springer, 2010, pp. 92–101
2010
Earlier work this paper cites.
X. Glorot, A. Bordes, and Y. Bengio, “Deep Sparse Rectifier Neural Networks,” in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , Jun. 2011, pp. 315–323. [Online]. Available: http://proceedings.mlr.press/v15/glorot11a.html
2011
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems , 2012, pp. 1097–1105
2012
Earlier work this paper cites.
P.-H. Pham, D. Jelaca, C. Farabet, B. Martini, Y. LeCun, and E. Culurciello, “Neuflow: Dataflow vision processing system-on-a-chip,” in 2012 IEEE 55th International Midwest Symposium on Circuits and Systems
2012
Earlier work this paper cites.
C. Brandli, R. Berner, M. Yang, S.-C. Liu, and T. Delbruck, “A 240 × \times 180 130 dB 3 μ \mu s latency global shutter spatiotemporal vision sensor,” IEEE Journal of Solid-State Circuits , vol. 49, no. 10, 2014
2014
Earlier work this paper cites.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014
Original
2014
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” arXiv preprint arXiv:1512.03385 , 2015
Original
2015
Earlier work this paper cites.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature , vol. 521, no. 7553, pp. 436–444, 2015
2015
Earlier work this paper cites.
M. Courbariaux, Y. Bengio, and J.-P. David, “Binaryconnect: Training deep neural networks with binary weights during propagations,” in Advances in Neural Information Processing Systems , 2015
2015
Earlier work this paper cites.
D. D. Lin, S. S. Talathi, and V. S. Annapureddy, “Fixed Point Quantization of Deep Convolutional Networks,” arXiv:1511.06393 , Nov. 2015. [Online]. Available: http://arxiv.org/abs/1511.06393
Original
2015
Earlier work this paper cites.

Similar

Z. Du, R. Fasthuber, T. Chen, P. Ienne, L. Li, T. Luo, X. Feng, Y. Chen, and O. Temam, “ShiDianNao,” ISCA 2015
2015
Cited alongside, same era.
E. Stromatias, D. Neil, M. Pfeiffer, F. Galluppi, S. B. Furber, and S.-C. Liu, “Robustness of spiking Deep Belief Networks to noise and reduced bit precision of neuro-inspired hardware platforms,” Frontiers in Neuroscience , vol. 9, Jul. 2015
2015
Cited alongside, same era.
M. Schaffner, F. K. Gürkaynak, A. Smolic, and L. Benini, “DRAM or no-DRAM?: Exploring Linear Solver Architectures for Image Domain Warping in 28 nm CMOS,” Proceedings of DATE , pp. 707–712, 2015
2015
Cited alongside, same era.
L. Cavigelli, D. Gschwend, C. Mayer, S. Willi, B. Muheim, and L. Benini, “Origami: A convolutional network accelerator,” in Proceedings of the 25th edition on Great Lakes Symposium on VLSI , 2015
2015
Cited alongside, same era.
M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi, “XNOR-net: ImageNet classification using binary convolutional neural networks,” in European Conference on Computer Vision . Springer, 2016
2016
Cited alongside, same era.
S. Zhou, Y. Wu, Z. Ni, X. Zhou, H. Wen, and Y. Zou, “DoReFa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients,” arXiv preprint arXiv:1606.06160 , 2016
Original
2016
Cited alongside, same era.
I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, “Quantized neural networks: Training neural networks with low precision weights and activations,” arXiv preprint arXiv:1609.07061 , 2016
Original
2016
Cited alongside, same era.
Y.-H. Chen, T. Krishna, J. Emer, and V. Sze, “Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks,” in IEEE International Solid-State Circuits Conference , 2016
2016
Cited alongside, same era.
B. Moons and M. Verhelst, “A 0.3-2.6 TOPS/W precision-scalable processor for real-time large-scale ConvNets,” IEEE Symposium on VLSI Circuits, Digest of Technical Papers , vol. 2016-Sept, pp. 1–2
2016
Cited alongside, same era.
S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz, and W. J. Dally, “EIE: Efficient Inference Engine on Compressed Deep Neural Network,” arXiv: 1602.01528 , 2016
2016
Cited alongside, same era.
S. Zhang, Z. Du, L. Zhang, H. Lan, S. Liu, L. Li, Q. Guo, T. Chen, and Y. Chen, “Cambricon-X: An accelerator for sparse neural networks,” in Proceedings of the Annual International Symposium on Microarchitecture, MICRO , vol. 2016-Dec. IEEE, oct 2016, pp. 1–12
2016
Cited alongside, same era.
J. Sim, J.-S. Park, M. Kim, D. Bae, Y. Choi, and L.-S. Kim, “14.6 A 1.42 TOPS/W deep convolutional neural network recognition processor for intelligent IoE systems,” in 2016 IEEE International Solid-State Circuits Conference (ISSCC) . IEEE, 2016, pp. 264–265
2016
Cited alongside, same era.

Then

P. Gysel, M. Motamedi, and S. Ghiasi, “Hardware-oriented approximation of convolutional neural networks,” arXiv preprint arXiv:1604.03168 , 2016
Original
2016
Later among the works it cites.
D. P. Moeys, F. Corradi, E. Kerr, P. Vance, G. Das, D. Neil, D. Kerr, and T. Delbruck, “Steering a predator robot using a mixed frame/event-driven convolutional neural network,” in 2016 IEEE Conf. on Event Based Control Communication and Signal Processing (EBCCSP 2016)
2016
Later among the works it cites.
A. Aimar, E. Calabrese, A. Rios-Navarro, R. Tapiador-Morales, I.-A. Lungu, F. Corradi, A. Linares-Barranco, and T. Delbruck. Roshambo NullHop CNN accelerator driven by DVS sensor, NIPS 2016. [Online]. Available: https://www.youtube.com/watch?v=KeLnZZYLexE
2016
Later among the works it cites.
A. Mishra, E. Nurvitadhi, J. J. Cook, and D. Marr, “WRPN: Wide Reduced-Precision Networks,” arXiv:1709.01134 [cs] , Sep. 2017, arXiv: 1709.01134. [Online]. Available: http://arxiv.org/abs/1709.01134
Original
2017
Closest in time.
B. Moons, R. Uytterhoeven, W. Dehaene, and M. Verhelst, “14.5 Envision: A 0.26-to-10TOPS/W subword-parallel dynamic-voltage-accuracy-frequency-scalable Convolutional Neural Network processor in 28nm FDSOI,” in 2017 IEEE International Solid-State Circuits Conference (ISSCC) , 2017, pp. 246–247
2017
Closest in time.
S. Yin, P. Ouyang, S. Tang, F. Tu, X. Li, L. Liu, and S. Wei, “A 1.06-to-5.09 TOPS/W reconfigurable hybrid-neural-network processor for deep learning applications,” in 2017 IEEE Symposium on VLSI Circuits
2017
Closest in time.
D. Shin, J. Lee, J. Lee, and H.-J. Yoo, “14.2 DNPU: An 8.1TOPS/W reconfigurable CNN-RNN processor for general-purpose deep neural networks,” in 2017 IEEE International Solid-State Circuits Conference (ISSCC) . IEEE, feb 2017, pp. 240–241
2017
Closest in time.
M. B. Milde, D. Neil, A. Aimar, T. Delbruck, and G. Indiveri, “ADaPTION: Toolbox and Benchmark for Training Convolutional Neural Networks with Reduced Numerical Precision Weights and Activation,” arXiv:1711.04713 , 2017
Original
2017
Closest in time.
I.-A. Lungu, F. Corradi, and T. Delbruck, “Live Demonstration: Convolutional Neural Network Driven by Dynamic Vision Sensor Playing RoShamBo,” in 2017 IEEE Symposium on Circuits and Systems (ISCAS 2017) , Baltimore, MD, USA, 2017
2017
Closest in time.
D. Neil, J. H. Lee, T. Delbruck, and S.-C. Liu, “Delta Networks for Optimized Recurrent Network Computation,” in PMLR , Jul. 2017, pp. 2584–2593. [Online]. Available: http://proceedings.mlr.press/v70/neil17a.html
2017
Closest in time.
C. Gao, D. Neil, E. Ceolini, S.-C. Liu, and T. Delbruck, “DeltaRNN: A Power-efficient RNN Accelerator,” in FPGA 18 , Monterey, CA, Feb. 2018
2017
Closest in time.
J. Lee, C. Kim, S. Kang, D. Shin, S. Kim, and H.-J. Yoo, “UNPU: a 50.6 TTOPS/W unified deep neural network accelerator with 1b-to-16b fully-variable weight bit-precision,” in 2018 IEEE International Solid-State Circuits Conference (ISSCC) . IEEE, feb 2018
2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…