Quantisation and Pruning for Neural Network Compression and Regularisation

E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy considerations for deep learning in NLP (2019),” arXiv preprint arXiv:1906.02243

Original

1906

Earlier work this paper cites.

D. E. Rumelhart, G. E. Hinton, R. J. Williams et al. , “Learning representations by back-propagating errors,” Cognitive modeling , vol. 5, no. 3, p. 1, 1988

1988

Earlier work this paper cites.

Y. LeCun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” in Advances in neural information processing systems , 1990, pp. 598–605

1990

Earlier work this paper cites.

B. Hassibi and D. G. Stork, “Second order derivatives for network pruning: Optimal brain surgeon,” in Advances in neural information processing systems , 1993, pp. 164–171

1993

Earlier work this paper cites.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Advances in neural information processing systems , 2012, pp. 1097–1105

2012

Earlier work this paper cites.

M. Denil, B. Shakibi, L. Dinh, N. De Freitas et al. , “Predicting parameters in deep learning,” in Advances in neural information processing systems , 2013, pp. 2148–2156

2013

Earlier work this paper cites.

A. Krizhevsky, V. Nair, and G. Hinton, “The CIFAR-10 dataset,” online: http://www. cs. toronto. edu/kriz/cifar. html , vol. 55, 2014

2014

Earlier work this paper cites.

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” The Journal of Machine Learning Research , vol. 15, no. 1, pp. 1929–1958, 2014

2014

Earlier work this paper cites.

L. Sifre, “Rigid-motion scattering for image classification,” Ph.D. dissertation, 2014

2014

Earlier work this paper cites.

S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167 , 2015

Original

2015

Cited alongside, same era.

S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural networks,” in Advances in neural information processing systems , 2015, pp. 1135–1143

2015

Cited alongside, same era.

W. Chen, J. Wilson, S. Tyree, K. Weinberger, and Y. Chen, “Compressing neural networks with the hashing trick,” in International Conference on Machine Learning , 2015, pp. 2285–2294

2015

Cited alongside, same era.

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 1–9

2015

Cited alongside, same era.

H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747 , 2017

Original

2017

Later among the works it cites.

J.-H. Luo, J. Wu, and W. Lin, “ThiNet: A filter level pruning method for deep neural network compression,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 5058–5066

2017

Later among the works it cites.

C. Louizos, M. Welling, and D. P. Kingma, “Learning sparse neural networks through L 0 {L}_{0} regularization,” in Proceedings of the International Conference on Learning Representations , 2017

2017

Later among the works it cites.

A. Paszke, S. Gross, S. Chintala, and G. Chanan, “Pytorch: Tensors and dynamic neural networks in python with strong gpu acceleration,” PyTorch: Tensors and dynamic neural networks in Python with strong GPU acceleration , vol. 6, 2017

2017

Later among the works it cites.