2017

Hello Edge: Keyword Spotting on Microcontrollers

Zhang, Yundong, Suda, Naveen, Lai, Liangzhen et al.

Understand

Keyword spotting (KWS) is a critical component for enabling speech based user interactions on smart devices.

It requires real-time response and high accuracy for good user experience.
Recently, neural networks have become an attractive choice for KWS architecture because of their superior accuracy compared to traditional speech processing algorithms.
Due to its always-on nature, KWS application has highly constrained power budget and typically runs on tiny microcontrollers with limited memory and compute capability.

Built on

Experimental, limited vocabulary, speech recognizer
C Teacher, H Kellett, and L Focht · 1967
Earlier work this paper cites.
Automatic recognition of keywords in unconstrained speech using hidden markov models
Jay G Wilpon, Lawrence R Rabiner, C-H Lee, and ER Goldman · 1990
Earlier work this paper cites.
A hidden markov model based keyword recognition system
Richard C Rose and Douglas B Paul · 1990
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Bidirectional recurrent neural networks
Mike Schuster and Kuldip K Paliwal · 1997
Earlier work this paper cites.
Learning precise timing with lstm recurrent networks
Felix A Gers, Nicol N Schraudolph, and Jürgen Schmidhuber · 2002
Earlier work this paper cites.
An application of recurrent neural networks to discriminative keyword spotting
Santiago Fernández, Alex Graves, and Jürgen Schmidhuber · 2007
Earlier work this paper cites.
Discriminative keyword spotting
Joseph Keshet, David Grangier, and Samy Bengio · 2009
Earlier work this paper cites.
Recurrent neural network based language model
Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur · 2010
Earlier work this paper cites.
Low-rank matrix factorization for deep neural network training with high-dimensional output targets
Tara N Sainath, Brian Kingsbury, Vikas Sindhwani, Ebru Arisoy, and Bhuvana Ramabhadran · 2013
Earlier work this paper cites.
Small-footprint keyword spotting using deep neural networks
Guoguo Chen, Carolina Parada, and Georg Heigold · 2014
Earlier work this paper cites.
Do deep nets really need to be deep?
Jimmy Ba and Rich Caruana · 2014
Earlier work this paper cites.

Similar

Long short-term memory recurrent neural network architectures for large scale acoustic modeling
Haşim Sak, Andrew Senior, and Françoise Beaufays · 2014
Cited alongside, same era.
Sequence to sequence learning with neural networks
Ilya Sutskever, Oriol Vinyals, and Quoc V Le · 2014
Cited alongside, same era.
Learning phrase representations using rnn encoder-decoder for statistical machine translation
Original
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio · 2014
Cited alongside, same era.
Empirical evaluation of gated recurrent neural networks on sequence modeling
Original
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio · 2014
Cited alongside, same era.
Adam: A method for stochastic optimization
Original
Diederik Kingma and Jimmy Ba · 2014
Cited alongside, same era.
Convolutional neural networks for small-footprint keyword spotting
Tara N Sainath and Carolina Parada · 2015
Cited alongside, same era.
Compressing deep neural networks using a rank-constrained topology
Preetum Nakkiran, Raziel Alvarez, Rohit Prabhavalkar, and Carolina Parada · 2015
Cited alongside, same era.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Cited alongside, same era.
Personalized speech recognition on mobile devices
Ian McGraw, Rohit Prabhavalkar, Raziel Alvarez, Montse Gonzalez Arenas, Kanishka Rao, David Rybach, Ouais Alsharif, Haşim Sak, Alexander Gruenstein, Françoise Beaufays, et al · 2016
Cited alongside, same era.
Max-pooling loss training of long short-term memory networks for small-footprint keyword spotting
Ming Sun, Anirudh Raju, George Tucker, Sankaran Panchapagesan, Gengshen Fu, Arindam Mandal, Spyros Matsoukas, Nikko Strom, and Shiv Vitaladevuni · 2016
Cited alongside, same era.
Model compression applied to small-footprint keyword spotting
George Tucker, Minhua Wu, Ming Sun, Sankaran Panchapagesan, Gengshen Fu, and Shiv Vitaladevuni · 2016
Cited alongside, same era.
Layer normalization
Original
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton · 2016
Cited alongside, same era.

Then

Xception: Deep learning with depthwise separable convolutions
Original
François Chollet · 2016
Later among the works it cites.
Tensorflow: Large-scale machine learning on heterogeneous distributed systems
Original
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al · 2016
Later among the works it cites.
Throughput-optimized opencl-based fpga accelerator for large-scale convolutional neural networks
Naveen Suda, Vikas Chandra, Ganesh Dasika, Abinash Mohanty, Yufei Ma, Sarma Vrudhula, Jae-sun Seo, and Yu Cao · 2016
Later among the works it cites.
Going deeper with embedded fpga platform for convolutional neural network
Jiantao Qiu, Jie Wang, Song Yao, Kaiyuan Guo, Boxun Li, Erjin Zhou, Jincheng Yu, Tianqi Tang, Ningyi Xu, Sen Song, et al · 2016
Later among the works it cites.
Dual path networks
Original
Yunpeng Chen, Jianan Li, Huaxin Xiao, Xiaojie Jin, Shuicheng Yan, and Jiashi Feng · 2017
Closest in time.
The microsoft 2017 conversational speech recognition system
Original
W. Xiong, L. Wu, F. Alleva, Jasha Droppo, X. Huang, and Andreas Stolcke · 2017
Closest in time.
Convolutional recurrent neural networks for small-footprint keyword spotting
Original
Sercan O Arik, Markus Kliegl, Rewon Child, Joel Hestness, Andrew Gibiansky, Chris Fougner, Ryan Prenger, and Adam Coates · 2017
Closest in time.
Speech commands: A public dataset for single-word speech recognition
Pete Warden · 2017
Closest in time.
Mobilenets: Efficient convolutional neural networks for mobile vision applications
Original
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam · 2017
Closest in time.
Shufflenet: An extremely efficient convolutional neural network for mobile devices
Original
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun · 2017
Closest in time.
Deep convolutional neural network inference with floating-point weights and fixed-point activations
Original
Liangzhen Lai, Naveen Suda, and Vikas Chandra · 2017
Closest in time.
CMSIS-NN: Efficient neural network kernels for Arm Cortex-M CPUs
Original
Liangzhen Lai, Naveen Suda, and Vikas Chandra · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…