2018

Deep-FSMN for Large Vocabulary Continuous Speech Recognition

Zhang, Shiliang, Lei, Ming, Yan, Zhijie et al.

Understand

In this paper, we present an improved feedforward sequential memory networks (FSMN) architecture, namely Deep-FSMN (DFSMN), by introducing skip connections between memory blocks in adjacent layers.

These skip connections enable the information flow across different layers and thus alleviate the gradient vanishing problem when building very deep structure.
As a result, DFSMN significantly benefits from these skip connections and deep structure.
We have compared the performance of DFSMN to BLSTM both with and without lower frame rate (LFR) on several large speech recognition tasks, including English and Mandarin.

Built on

“Phoneme recognition using time-delay neural networks,”
Alexander Waibel, Toshiyuki Hanazawa, Geoffrey Hinton, Kiyohiro Shikano, and Kevin J Lang, · 1989
Earlier work this paper cites.
“Finding structure in time,”
J. L. Elman, · 1990
Earlier work this paper cites.
“Backpropagation through time: what it does and how to do it,”
P. J. Werbos, · 1990
Earlier work this paper cites.
“Learning long-term dependencies with gradient descent is difficult,”
Y. Bengio, P. Simard, and P. Frasconi, · 1994
Earlier work this paper cites.
“Long short-term memory,”
Sepp Hochreiter and Jürgen Schmidhuber, · 1997
Earlier work this paper cites.
“The fisher corpus: a resource for the next generations of speech-to-text.,”
Christopher Cieri, David Miller, and Kevin Walker, · 2004
Earlier work this paper cites.
“Acoustic modeling using deep belief networks,”
Abdel Rahman Mohamed, George E Dahl, and Geoffrey Hinton, · 2012
Earlier work this paper cites.
“Context-dependent pre-trained deep neural networks for large vocabulary speech recognition,”
George E Dahl, Dong Yu, Li Deng, and Alex Acero, · 2012
Earlier work this paper cites.
“Applying convolutional neural networks concepts to hybrid NN-HMM model for speech recognition,”
Ossama Abdel-Hamid, Abdel Rahman Mohamed, Hui Jiang, and Gerald Penn, · 2012
Earlier work this paper cites.
“Speech recognition with deep recurrent neural networks,”
A. Graves, A. Mohamed, and G. E. Hinton, · 2013
Earlier work this paper cites.
“Convolutional neural networks for speech recognition,”
Ossama Abdel Hamid, Abdel Rahman Mohamed, Hui Jiang, Li Deng, Gerald Penn, and Dong Yu, · 2014
Earlier work this paper cites.

Similar

“Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition,”
Original
H. Sak, A. Senior, and F. Beaufays, · 2014
Cited alongside, same era.
“Unfolded recurrent neural networks for speech recognition,”
G. Saon, H. Soltau, A. Emami, and M. Picheny, · 2014
Cited alongside, same era.
“Grid long short-term memory,”
Original
Nal Kalchbrenner, Ivo Danihelka, and Alex Graves, · 2015
Cited alongside, same era.
“A time delay neural network architecture for efficient modeling of long temporal contexts,”
Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur, · 2015
Cited alongside, same era.
“Reverberation robust acoustic modeling using i-vectors with time delay neural networks,”
Vijayaditya Peddinti, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, · 2015
Cited alongside, same era.
“Feedforward sequential memory neural networks without recurrent feedback,”
Original
Shiliang Zhang, Hui Jiang, Si Wei, and Lirong Dai, · 2015
Cited alongside, same era.
“Feedforward sequential memory networks: A new structure to learn long-term dependency,”
Original
Shiliang Zhang, Cong Liu, Hui Jiang, Si Wei, Lirong Dai, and Yu Hu, · 2015
Cited alongside, same era.
“Highway networks,”
Original
Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber, · 2015
Cited alongside, same era.
“Rectified linear neural networks with tied-scalar regularization for lvcsr,”
Shiliang Zhang, Hui Jiang, Si Wei, and Li-Rong Dai, · 2015
Cited alongside, same era.
“Modeling time-frequency patterns with LSTM vs. convolutional architectures for LVCSR tasks.,”
Tara N Sainath and Bo Li, · 2016
Cited alongside, same era.

Then

“Compact feedforward sequential memory networks for large vocabulary continuous speech recognition.,”
Shiliang Zhang, Hui Jiang, Shifu Xiong, Si Wei, and Li-Rong Dai, · 2016
Later among the works it cites.
“Deep residual learning for image recognition,”
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, · 2016
Later among the works it cites.
“Highway long short-term memory rnns for distant speech recognition,”
Yu Zhang, Guoguo Chen, Dong Yu, Kaisheng Yaco, Sanjeev Khudanpur, and James Glass, · 2016
Later among the works it cites.
“Lower frame rate neural network acoustic models.,”
Golan Pundak and Tara N Sainath, · 2016
Later among the works it cites.
“Improving latency-controlled blstm acoustic models for online speech recognition,”
Shaofei Xue and Zhijie Yan, · 2016
Later among the works it cites.
“Wavenet: A generative model for raw audio,”
Original
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu, · 2016
Later among the works it cites.
“Achieving human parity in conversational speech recognition,”
Original
Wayne Xiong, Jasha Droppo, Xuedong Huang, Frank Seide, Mike Seltzer, Andreas Stolcke, Dong Yu, and Geoffrey Zweig, · 2016
Later among the works it cites.
“Scalable training of deep learning machines by incremental block training with intra-block parallel optimization and blockwise model-update filtering,”
Kai Chen and Qiang Huo, · 2016
Later among the works it cites.
“Nonrecurrent neural structure for long-term dependence,”
Shiliang Zhang, Cong Liu, Hui Jiang, Si Wei, Lirong Dai, and Yu Hu, · 2017
Later among the works it cites.
“Highway-LSTM and recurrent highway networks for speech recognition,”
Golan Pundak and Tara Sainath, · 2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…