Run-Time Efficient RNN Compression for Inference on Edge Devices

Measuring scheduling efficiency of RNNs for NLP applications

Original

Urmish Thakker, Ganesh Dasika, Jesse G. Beu, and Matthew Mattina. 2019b · 1904

Earlier work this paper cites.

Compressing RNNs for IoT devices by 15-38x using Kronecker Products

Original

Urmish Thakker, Jesse G. Beu, Dibakar Gope, Chu Zhou, Igor Fedorov, Ganesh Dasika, and Matthew Mattina. 2019a · 1906

Earlier work this paper cites.

Ternary MobileNets via Per-Layer Hybrid Filter Banks

Dibakar Gope, Jesse Beu, Urmish Thakker, and Matthew Mattina. 2019 · 1911

Earlier work this paper cites.

Compressing Language Models using Doped Kronecker Products

Urmish Thakker, Paul Whatmough, Matthew Mattina, and Jesse Beu. 2020 · 2001

Earlier work this paper cites.

Collecting complex activity datasets in highly rich networked sensor environments. In 2010 Seventh International Conference on Networked Sensing Systems (INSS) . 233–240

D. Roggen, A. Calatroni, M. Rossi, T. Holleczek, K. Förster, G. Tröster, P. Lukowicz, D. Bannach, G. Pirkl, A. Ferscha, J. Doppler, C. Holzmann, M. Kurz, G. Holl, R. Chavarriaga, H. Sagha, H. Bayati, M. Creatura, and J. d. R. Millàn. 2010 · 2010

Earlier work this paper cites.

Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing . 6655–6659

T. N. Sainath, B. Kingsbury, V. Sindhwani, E. Arisoy, and B. Ramabhadran. 2013 · 2013

Earlier work this paper cites.

Recurrent Neural Network Regularization

Original

Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014 · 2014

Earlier work this paper cites.

Structured Transforms for Small-Footprint Deep Learning

Vikas Sindhwani, Tara Sainath, and Sanjiv Kumar. 2015 · 2015

Cited alongside, same era.

Deep, convolutional, and recurrent models for human activity recognition using wearables

Nils Y Hammerla, Shane Halloran, and Thomas Ploetz. 2016 · 2016

Cited alongside, same era.

Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

Song Han, Huizi Mao, and William J Dally. 2016 · 2016

Cited alongside, same era.

Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition

Francisco Javier Ordóñez and Daniel Roggen. 2016 · 2016

Cited alongside, same era.

CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-circulant Weight Matrices. In Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO-50 ’17) . ACM, New York, NY, USA, 395–408

Caiwen Ding, Siyu Liao, Yanzhi Wang, Zhe Li, Ning Liu, Youwei Zhuo, Chao Wang, Xuehai Qian, Yu Bai, Geng Yuan, Xiaolong Ma, Yipeng Zhang, Jian Tang, Qinru Qiu, Xue Lin, and Bo Yuan. 2017 · 2017

Compressing recurrent neural network with tensor train. In Neural Networks (IJCNN), 2017 International Joint Conference on . IEEE, 4451–4458

Andros Tjandra, Sakriani Sakti, and Satoshi Nakamura. 2017 · 2017

Later among the works it cites.

To prune, or not to prune: exploring the efficacy of pruning for model compression

Original

Michael Zhu and Suyog Gupta. 2017 · 2017

Later among the works it cites.

Adaptive Mixture of Low-Rank Factorizations for Compact Neural Modeling

Ting Chen, Ji Lin, Tian Lin, Song Han, Chong Wang, and Denny Zhou. 2018 · 2018

Later among the works it cites.

Structured Weight Matrices-Based Hardware Accelerators in Deep Neural Networks: FPGAs and ASICs. In Proceedings of the 2018 on Great Lakes Symposium on VLSI (GLSVLSI ’18) . ACM, New York, NY, USA, 353–358

Caiwen Ding, Ao Ren, Geng Yuan, Xiaolong Ma, Jiayu Li, Ning Liu, Bo Yuan, and Yanzhi Wang. 2018 · 2018

Later among the works it cites.

Efficient Recurrent Neural Networks using Structured Matrices in FPGAs