Fetching the paper…
Reading the bibliography…
We study the neural network (NN) compression problem, viewing the tension between the compression ratio and NN performance through the lens of rate-distortion theory.
The state of sparsity in deep neural networks
Gale, T., Elsen, E., and Hooker, S. (2019) · 1902
Earlier work this paper cites.
Morcos, A. S., Yu, H., Paganini, M., and Tian, Y. (2019) · 1906
Earlier work this paper cites.
Sparse networks from scratch: Faster training without losing performance
Dettmers, T. and Zettlemoyer, L. (2019) · 1907
Earlier work this paper cites.
Advances and open problems in federated learning
Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., et al. (2019) · 1912
Earlier work this paper cites.
A mathematical theory of communication
Shannon, C. E. (1948) · 1948
Earlier work this paper cites.
A method for the construction of minimum-redundancy codes
Huffman, D. A. (1952) · 1952
Earlier work this paper cites.
Coding theorems for a discrete source with a fidelity criterion
Shannon, C. E. (1959) · 1959
Earlier work this paper cites.
Run-length encodings (corresp.)
Golomb, S. (1966) · 1966
Earlier work this paper cites.
Optimal source codes for geometrically distributed integer alphabets (corresp.)
Gallager, R. and Van Voorhis, D. (1975) · 1975
Earlier work this paper cites.
Hierarchical coding of discrete sources
Koshelev, V. N. (1980) · 1980
Earlier work this paper cites.
Optimal Brain Damage
Cun, Y. L., Denker, J. S., and Solla, S. A. (1990) · 1990
Earlier work this paper cites.
Successive refinement of information
Equitz, W. H. and Cover, T. M. (1991) · 1991
Earlier work this paper cites.
Image compression using the 2-d wavelet transform
Lewis, A. S. and Knowles, G. (1992) · 1992
Earlier work this paper cites.
Optimal brain surgeon: Extensions and performance comparisons
Hassibi, B., Stork, D. G., Wolff, G., and Watanabe, T. (1993) · 1993
Earlier work this paper cites.
The exponential distribution in information theory
Verdu, S. (1996) · 1996
Earlier work this paper cites.
Gradient-based learning applied to document recognition
LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998) · 1998
Earlier work this paper cites.
Jpeg2000: Image compression fundamentals, standards and practice
Rabbani, M. (2002) · 2002
Earlier work this paper cites.
Rate-distortion theory
Berger, T. (2003) · 2003
Earlier work this paper cites.
What is the state of neural network pruning?
Blalock, D., Ortiz, J. J. G., Frankle, J., and Guttag, J. (2020) · 2003
Earlier work this paper cites.
Data compression: the complete reference
Salomon, D. (2004) · 2004
Earlier work this paper cites.
Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing)
Cover, T. M. and Thomas, J. A. (2006) · 2006
Earlier work this paper cites.
The minimum description length principle
Grünwald, P. D. and Grunwald, A. (2007) · 2007
Earlier work this paper cites.
ImageNet: A Large-Scale Hierarchical Image Database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009) · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G., et al. (2009) · 2009
Earlier work this paper cites.
Transform quantization for cnn compression
Young, S. I., Zhe, W., Taubman, D., and Girod, B. (2020) · 2009
Earlier work this paper cites.
Characterising bias in compressed models
Hooker, S., Moorosi, N., Clark, G., Bengio, S., and Denton, E. (2020) · 2010
Earlier work this paper cites.
Mnist handwritten digit database
LeCun, Y., Cortes, C., and Burges, C. (2010) · 2010
Earlier work this paper cites.
Low-rank matrix factorization for deep neural network training with high-dimensional output targets
Sainath, T. N., Kingsbury, B., Sindhwani, V., Arisoy, E., and Ramabhadran, B. (2013) · 2013
Earlier work this paper cites.
Densenet: Implementing efficient convnet descriptor pyramids
Iandola, F., Moskewicz, M., Karayev, S., Girshick, R., Darrell, T., and Keutzer, K. (2014) · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Simonyan, K. and Zisserman, A. (2014) · 2014
Cited alongside, same era.
Lossy compression via sparse linear regression: Computationally efficient encoding and decoding
Venkataramanan, R., Sarkar, T., and Tatikonda, S. (2014) · 2014
Cited alongside, same era.
Distilling the knowledge in a neural network
Hinton, G., Vinyals, O., and Dean, J. (2015) · 2015
Cited alongside, same era.
Training cnns with low-rank filters for efficient image classification
Ioannou, Y., Robertson, D., Shotton, J., Cipolla, R., and Criminisi, A. (2015) · 2015
Cited alongside, same era.
End-to-end optimized image compression
Ballé, J., Laparra, V., and Simoncelli, E. P. (2016) · 2016
Cited alongside, same era.
Gradient sparsification for communication-efficient distributed optimization
Wangni, J., Wang, J., Liu, J., and Zhang, T. (2018) · 2018
Later among the works it cites.
Nisp: Pruning networks using neuron importance score propagation
Yu, R., Li, A., Chen, C.-F., Lai, J.-H., Morariu, V. I., Han, X., Gao, M., Lin, C.-Y., and Davis, L. S. (2018) · 2018
Later among the works it cites.
The lottery ticket hypothesis: Finding sparse, trainable neural networks
Frankle, J. and Carbin, M. (2019) · 2019
Later among the works it cites.
Rate distortion for model compression: From theory to practice
Gao, W., Liu, Y.-H., Wang, C., and Oh, S. (2019) · 2019
Later among the works it cites.
Minimal random code learning: Getting bits back from compressed model parameters
Havasi, M., Peharz, R., and Hernández-Lobato, J. M. (2019) · 2019
Later among the works it cites.
Learning to quantize deep networks by optimizing quantization intervals with task loss
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Dynamic network surgery for efficient dnns
Guo, Y., Yao, A., and Chen, Y. (2016) · 2016
Cited alongside, same era.
Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding
Han, S., Mao, H., and Dally, W. J. (2016) · 2016
Cited alongside, same era.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J. (2016) · 2016
Cited alongside, same era.
Federated learning: Strategies for improving communication efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtarik, P., Suresh, A. T., and Bacon, D. (2016) · 2016
Cited alongside, same era.
Li, F., Zhang, B., and Liu, B. (2016) · 2016
Cited alongside, same era.
Pruning convolutional neural networks for resource efficient inference
Molchanov, P., Tyree, S., Karras, T., Aila, T., and Kautz, J. (2016) · 2016
Cited alongside, same era.
Strong successive refinability and rate-distortion-complexity tradeoff
No, A., Ingber, A., and Weissman, T. (2016) · 2016
Cited alongside, same era.
Jung, S., Son, C., Lee, S., Son, J., Han, J.-J., Kwak, Y., Hwang, S. J., and Choi, C. (2019) · 2019
Later among the works it cites.
Towards optimal structured cnn pruning via generative adversarial learning
Lin, S., Ji, R., Yan, C., Zhang, B., Cao, L., Ye, Q., Huang, F., and Doermann, D. (2019) · 2019
Later among the works it cites.
Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization
Mostafa, H. and Wang, X. (2019) · 2019
Later among the works it cites.
Scalable model compression by entropy penalized reparameterization
Oktay, D., Ballé, J., Singh, S., and Shrivastava, A. (2019) · 2019
Later among the works it cites.
Pytorch: An imperative style, high-performance deep learning library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. (2019) · 2019
Later among the works it cites.
Collaborative channel pruning for deep networks
Peng, H., Wu, J., Chen, S., and Huang, J. (2019) · 2019
Later among the works it cites.
Efficientnet: Rethinking model scaling for convolutional neural networks
Tan, M. and Le, Q. (2019) · 2019
Later among the works it cites.
Autoprune: Automatic network pruning by regularizing auxiliary parameters
Xiao, X., Wang, Z., and Rajasekaran, S. (2019) · 2019
Later among the works it cites.
Variational convolutional neural network pruning
Zhao, C., Ni, B., Zhang, J., Zhao, Q., Zhang, W., and Tian, Q. (2019) · 2019
Later among the works it cites.
rtop-k: A statistical estimation approach to distributed sgd
Barnes, L. P., Inan, H. A., Isik, B., and Özgür, A. (2020) · 2020
Later among the works it cites.
Universal deep neural network compression
Choi, Y., El-Khamy, M., and Lee, J. (2020) · 2020
Later among the works it cites.
Fast sparse convnets
Elsen, E., Dukhan, M., Gale, T., and Simonyan, K. (2020) · 2020
Later among the works it cites.
Rigging the lottery: Making all tickets winners
Evci, U., Gale, T., Menick, J., Castro, P. S., and Elsen, E. (2020) · 2020
Later among the works it cites.
Low-rank compression of neural nets: Learning the rank of each layer
Idelbayev, Y. and Carreira-Perpinan, M. A. (2020) · 2020
Later among the works it cites.
Lookahead: A far-sighted alternative of magnitude-based pruning
Park, S., Lee, J., Mo, S., and Shin, J. (2020) · 2020
Later among the works it cites.
Comparing fine-tuning and rewinding in neural network pruning
Renda, A., Frankle, J., and Carbin, M. (2020) · 2020
Later among the works it cites.
Deepcabac: A universal compression algorithm for deep neural networks
Wiedemann, S., Kirchhoffer, H., Matlage, S., Haase, P., Marban, A., Marinč, T., Neumann, D., Nguyen, T., Schwarz, H., Wiegand, T., Marpe, D., and Samek, W. (2020) · 2020
Later among the works it cites.
Long live the lottery: The existence of winning tickets in lifelong learning
Chen, T., Zhang, Z., Liu, S., Chang, S., and Wang, Z. (2021) · 2021
Closest in time.
Optimal quantization using scaled codebook
Idelbayev, Y., Molchanov, P., Shen, M., Yin, H., Carreira-Perpinan, M. A., and Alvarez, J. M. (2021) · 2021
Closest in time.
Neural network compression for noisy storage devices
Isik, B., Choi, K., Zheng, X., Weissman, T., Ermon, S., Wong, H. S. P., and Alaghi, A. (2021) · 2021
Closest in time.
Layer-adaptive sparsity for the magnitude-based pruning
Lee, J., Park, S., Mo, S., Ahn, S., and Shin, J. (2021) · 2021
Closest in time.
Training with quantization noise for extreme model compression
Stock, P., Fan, A., Graham, B., Grave, E., Gribonval, R., Jegou, H., and Joulin, A. (2021) · 2021
Closest in time.
Rate-distortion optimized coding for efficient cnn compression
Zhe, W., Lin, J., Aly, M. S., Young, S., Chandrasekhar, V., and Girod, B. (2021) · 2021
Closest in time.