Built on

A. Acero, “Acoustical and environmental robustness in automatic speech recognition,” in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing , 1990
1990
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation , vol. 9, no. 8, pp. 1735–1780, 1997
1997
Earlier work this paper cites.
A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks,” in Proc. Int. Conf. Machine Learning , 2006, pp. 369–376
2006
Earlier work this paper cites.
V. Vanhoucke, A. Senior, and M. Z. Mao, “Improving the speed of neural networks on CPUs,” in Proc. NIPS Workshop Deep Learning and Unsupervised Feature Learning , vol. 1, 2011, p. 4
2011
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Proc. Advances in Neural Information Processing Systems , 2012, pp. 1097–1105
2012
Earlier work this paper cites.
D. Yu, F. Seide, G. Li, and L. Deng, “Exploiting sparseness in deep neural networks for large vocabulary speech recognition,” in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing , 2012, pp. 4409–4412
2012
Earlier work this paper cites.
A. Graves, N. Jaitly, and A.-R. Mohamed, “Hybrid speech recognition with deep bidirectional LSTM,” in Proc. IEEE Workshop Automatic Speech Recognition and Understanding , 2013, pp. 273–278
2013
Earlier work this paper cites.
A. Graves, A.-R. Mohamed, and G. Hinton, “Speech recognition with deep recurrent neural networks,” in Proc. Int. Conf. Acoustics, Speech, and Signal Processing , 2013, pp. 6645–6649
2013
Earlier work this paper cites.
R. Pascanu, C. Gulcehre, K. Cho, and Y. Bengio, “How to construct deep recurrent neural networks,” arXiv preprint arXiv:1312.6026 , 2013
Original
2013
Earlier work this paper cites.
M. Denil, B. Shakibi, L. Dinh, N. De Freitas et al. , “Predicting parameters in deep learning,” in Proc. Advances in Neural Information Processing Systems , 2013, pp. 2148–2156
2013
Earlier work this paper cites.
I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Proc. Advances in Neural Information Processing Systems , 2014, pp. 3104–3112
2014
Earlier work this paper cites.
A. Hannun, C. Case, J. Casper, B. Catanzaro, G. Diamos, E. Elsen, R. Prenger, S. Satheesh, S. Sengupta, A. Coates et al. , “Deep Speech: Scaling up end-to-end speech recognition,” arXiv preprint arXiv:1412.5567 , 2014
Original
2014
Earlier work this paper cites.
W. Zaremba, I. Sutskever, and O. Vinyals, “Recurrent neural network regularization,” arXiv preprint arXiv:1409.2329 , 2014
Original
2014
Earlier work this paper cites.
E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, “Exploiting linear structure within convolutional networks for efficient evaluation,” in Proc. Advances in Neural Information Processing Systems , 2014, pp. 1269–1277
2014
Earlier work this paper cites.

Similar

M. Jaderberg, A. Vedaldi, and A. Zisserman, “Speeding up convolutional neural networks with low rank expansions,” arXiv preprint arXiv:1405.3866 , 2014
Original
2014
Cited alongside, same era.
Y. Gong, L. Liu, M. Yang, and L. Bourdev, “Compressing deep convolutional networks using vector quantization,” arXiv preprint arXiv:1412.6115 , 2014
Original
2014
Cited alongside, same era.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014
Original
2014
Cited alongside, same era.
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft COCO: Common objects in context,” in Proc. European Conf. Computer Vision , 2014, pp. 740–755
2014
Cited alongside, same era.

Then

Y. Zhang, G. Chen, D. Yu, K. Yaco, S. Khudanpur, and J. Glass, “Highway long short-term memory RNNs for distant speech recognition,” in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing , 2016, pp. 5755–5759
2016
Later among the works it cites.
Z. Lu, V. Sindhwani, and T. N. Sainath, “Learning compact recurrent neural networks,” in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing , 2016, pp. 5960–5964
2016
Later among the works it cites.
H. Zen, Y. Agiomyrgiannakis, N. Egberts, F. Henderson, and P. Szczepaniak, “Fast, compact, and high quality LSTM-RNN based statistical parametric speech synthesizers for mobile devices,” arXiv preprint arXiv:1606.06061 , 2016
Original
2016
Later among the works it cites.
S. Han, J. Pool, S. Narang, H. Mao, S. Tang, E. Elsen, B. Catanzaro, J. Tran, and W. J. Dally, “DSD: Regularizing deep neural networks with dense-sparse-dense training flow,” arXiv preprint arXiv:1607.04381 , 2016
Original
2016
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2018

Grow and Prune Compact, Fast, and Accurate LSTMs

Dai, Xiaoliang, Yin, Hongxu, Jha, Niraj K.

Understand

Long short-term memory (LSTM) has been widely used for sequential data modeling.

Researchers have increased LSTM depth by stacking LSTM cells to improve performance.
This incurs model redundancy, increases run-time delay, and makes the LSTMs more prone to overfitting.
To address these problems, we propose a hidden-layer LSTM (H-LSTM) that adds hidden layers to LSTM's original one level non-linear control gates.

Built on

A. Acero, “Acoustical and environmental robustness in automatic speech recognition,” in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing , 1990
1990
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation , vol. 9, no. 8, pp. 1735–1780, 1997
1997
Earlier work this paper cites.
A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks,” in Proc. Int. Conf. Machine Learning , 2006, pp. 369–376
2006
Earlier work this paper cites.
V. Vanhoucke, A. Senior, and M. Z. Mao, “Improving the speed of neural networks on CPUs,” in Proc. NIPS Workshop Deep Learning and Unsupervised Feature Learning , vol. 1, 2011, p. 4
2011
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Proc. Advances in Neural Information Processing Systems , 2012, pp. 1097–1105
2012
Earlier work this paper cites.
D. Yu, F. Seide, G. Li, and L. Deng, “Exploiting sparseness in deep neural networks for large vocabulary speech recognition,” in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing , 2012, pp. 4409–4412
2012
Earlier work this paper cites.
A. Graves, N. Jaitly, and A.-R. Mohamed, “Hybrid speech recognition with deep bidirectional LSTM,” in Proc. IEEE Workshop Automatic Speech Recognition and Understanding , 2013, pp. 273–278
2013
Earlier work this paper cites.
A. Graves, A.-R. Mohamed, and G. Hinton, “Speech recognition with deep recurrent neural networks,” in Proc. Int. Conf. Acoustics, Speech, and Signal Processing , 2013, pp. 6645–6649
2013
Earlier work this paper cites.
R. Pascanu, C. Gulcehre, K. Cho, and Y. Bengio, “How to construct deep recurrent neural networks,” arXiv preprint arXiv:1312.6026 , 2013
Original
2013
Earlier work this paper cites.
M. Denil, B. Shakibi, L. Dinh, N. De Freitas et al. , “Predicting parameters in deep learning,” in Proc. Advances in Neural Information Processing Systems , 2013, pp. 2148–2156
2013
Earlier work this paper cites.
I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Proc. Advances in Neural Information Processing Systems , 2014, pp. 3104–3112
2014
Earlier work this paper cites.
A. Hannun, C. Case, J. Casper, B. Catanzaro, G. Diamos, E. Elsen, R. Prenger, S. Satheesh, S. Sengupta, A. Coates et al. , “Deep Speech: Scaling up end-to-end speech recognition,” arXiv preprint arXiv:1412.5567 , 2014
Original
2014
Earlier work this paper cites.
W. Zaremba, I. Sutskever, and O. Vinyals, “Recurrent neural network regularization,” arXiv preprint arXiv:1409.2329 , 2014
Original
2014
Earlier work this paper cites.
E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, “Exploiting linear structure within convolutional networks for efficient evaluation,” in Proc. Advances in Neural Information Processing Systems , 2014, pp. 1269–1277
2014
Earlier work this paper cites.

Similar

M. Jaderberg, A. Vedaldi, and A. Zisserman, “Speeding up convolutional neural networks with low rank expansions,” arXiv preprint arXiv:1405.3866 , 2014
Original
2014
Cited alongside, same era.
Y. Gong, L. Liu, M. Yang, and L. Bourdev, “Compressing deep convolutional networks using vector quantization,” arXiv preprint arXiv:1412.6115 , 2014
Original
2014
Cited alongside, same era.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014
Original
2014
Cited alongside, same era.
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft COCO: Common objects in context,” in Proc. European Conf. Computer Vision , 2014, pp. 740–755
2014
Cited alongside, same era.

Then

Y. Zhang, G. Chen, D. Yu, K. Yaco, S. Khudanpur, and J. Glass, “Highway long short-term memory RNNs for distant speech recognition,” in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing , 2016, pp. 5755–5759
2016
Later among the works it cites.
Z. Lu, V. Sindhwani, and T. N. Sainath, “Learning compact recurrent neural networks,” in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing , 2016, pp. 5960–5964
2016
Later among the works it cites.
H. Zen, Y. Agiomyrgiannakis, N. Egberts, F. Henderson, and P. Szczepaniak, “Fast, compact, and high quality LSTM-RNN based statistical parametric speech synthesizers for mobile devices,” arXiv preprint arXiv:1606.06061 , 2016
Original
2016
Later among the works it cites.
S. Han, J. Pool, S. Narang, H. Mao, S. Tang, E. Elsen, B. Catanzaro, J. Tran, and W. J. Dally, “DSD: Regularizing deep neural networks with dense-sparse-dense training flow,” arXiv preprint arXiv:1607.04381 , 2016
Original
2016
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014

Original

2014

Cited alongside, same era.

K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” arXiv preprint arXiv:1406.1078 , 2014

Original

2014

Cited alongside, same era.

A. Karpathy and L. Fei-Fei, “Deep visual-semantic alignments for generating image descriptions,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition , 2015, pp. 3128–3137

2015

Cited alongside, same era.

W. Chen, J. Wilson, S. Tyree, K. Weinberger, and Y. Chen, “Compressing neural networks with the hashing trick,” in Proc. Int. Conf. Machine Learning , 2015, pp. 2285–2294

2015

Cited alongside, same era.

S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” in Proc. Advances in Neural Information Processing Systems , 2015, pp. 1135–1143

2015

Cited alongside, same era.

H. Zen and H. Sak, “Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis,” in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing , 2015, pp. 4470–4474

2015

Cited alongside, same era.

O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, “Show and tell: A neural image caption generator,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition , 2015, pp. 3156–3164

2015

Cited alongside, same era.

R. Vedantam, C. Lawrence Zitnick, and D. Parikh, “CIDEr: Consensus-based image description evaluation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition , 2015, pp. 4566–4575

2015

Cited alongside, same era.

A. Karpathy, “Image captioning in Torch,” https://github.com/karpathy/neuraltalk2 , 2016

2016