Retraining-Based Iterative Weight Quantization for Deep Neural Networks

Building a large annotated corpus of English: The Penn Treebank

M. P. Marcus, M. A. Marcinkiewicz, and B. Santorini · 1993

Earlier work this paper cites.

Simplifying neural nets by discovering flat minima

S. Hochreiter and J. Schmidhuber · 1995

Earlier work this paper cites.

Long short-term memory

S. Hochreiter and J. Schmidhuber · 1997

Earlier work this paper cites.

Low-rank matrix factorization for deep neural network training with high-dimensional output targets

T. N. Sainath, B. Kingsbury, V. Sindhwani, E. Arisoy, and B. Ramabhadran · 2013

Earlier work this paper cites.

Restructuring of deep neural network acoustic models with singular value decomposition

J. Xue, J. Li, and Y. Gong · 2013

Earlier work this paper cites.

Learning longer memory in recurrent neural networks

Original

T. Mikolov, A. Joulin, S. Chopra, M. Mathieu, and M. Ranzato · 2014

Earlier work this paper cites.

Recurrent neural network regularization

Original

W. Zaremba, I. Sutskever, and O. Vinyals · 2014

Earlier work this paper cites.

The loss surfaces of multilayer networks

A. Choromanska, M. Henaff, M. Mathieu, G. B. Arous, and Y. LeCun · 2015

Earlier work this paper cites.

BinaryConnect: Training deep neural networks with binary weights during propagations

M. Courbariaux, Y. Bengio, and J.-P. David · 2015

Earlier work this paper cites.

Learning both weights and connections for efficient neural networks

S. Han, J. Pool, J. Tran, and W. J. Dally · 2015

Earlier work this paper cites.

Distilling the knowledge in a neural network

G. Hinton, O. Vinyals, and J. Dean · 2015

Cited alongside, same era.

Deep learning

Y. LeCun, Y. Bengio, and G. E. Hinton · 2015

Cited alongside, same era.

Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding

S. Han, H. Mao, and W. J. Dally · 2016

Cited alongside, same era.

Quantized neural networks: training neural networks with low precision weights and activations

Original

I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio · 2016

Cited alongside, same era.

XNOR-Net: Imagenet classification using binary convolutional neural networks

M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi · 2016

Cited alongside, same era.

DoReFa-Net: training low bitwidth convolutional neural networks with low bitwidth gradients

Data noising as smoothing in neural network language models

Z. Xie, S. I. Wang, J. Li, D. Lévy, A. Nie, D. Jurafsky, and A. Y. Ng · 2017

Later among the works it cites.

Balanced quantization: An effective and efficient approach to quantized neural networks

S. Zhou, Y. Wang, H. Wen, Q. He, and Y. Zou · 2017

Later among the works it cites.

Trained ternary quantization

C. Zhu, S. Han, H. Mao, and W. J. Dally · 2017

Later among the works it cites.

Apaptive quantization of neural networks

S. Khoram and J. Li · 2018

Closest in time.

Viterbi-based pruning for sparse matrix with fixed and high index compression ratio

D. Lee, D. Ahn, T. Kim, P. I. Chuang, and J.-J. Kim · 2018

Closest in time.

Training wide residual networks for deployment using a single bit for each weight

M. D. McDonnell · 2018

Closest in time.