Understand
The performance of Deep Neural Networks (DNNs) keeps elevating in recent years with increasing network depth and width.
- To enable DNNs on edge devices like mobile phones, researchers proposed several network compression methods including pruning, quantization and factorization.
- Among the factorization-based approaches, low-rank approximation has been widely adopted because of its solid theoretical rationale and efficient implementations.
- Several previous works attempted to directly approximate a pre-trained model by low-rank decomposition; however, small approximation errors in parameters can ripple a large prediction loss.
Built on
Characterization of the subdifferential of some matrix norms
G. A. Watson · 1992
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
A. Krizhevsky and G. Hinton · 2009
Earlier work this paper cites.
Efficient and practical stochastic subgradient descent for nuclear norm regularization
H. Avron, S. Kale, S. P. Kasiviswanathan, and V. Sindhwani · 2012
Earlier work this paper cites.
Exploiting linear structure within convolutional networks for efficient evaluation
E. Denton, W. Zaremba, J. Bruna, Y. Lecun, and R. Fergus · 2014
Earlier work this paper cites.
Speeding up convolutional neural networks with low rank expansions
M. Jaderberg, A. Vedaldi, and A. Zisserman · 2014
Earlier work this paper cites.
Similar
Compressing neural networks with the hashing trick
W. Chen, J. Wilson, S. Tyree, K. Weinberger, and Y. Chen · 2015
Cited alongside, same era.
Faster r-cnn: Towards real-time object detection with region proposal networks
S. Ren, K. He, R. B. Girshick, and J. Sun · 2015
Cited alongside, same era.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Cited alongside, same era.
Pruning filters for efficient convnets
H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf · 2016
Cited alongside, same era.
Accelerating very deep convolutional networks for classification and detection
X. Zhang, J. Zou, K. He, and J. Sun · 2016
Cited alongside, same era.
Compression-aware training of deep networks
J. M. Alvarez and M. Salzmann · 2017
Cited alongside, same era.
Then
Channel pruning for accelerating very deep neural networks
Y. He, X. Zhang, and J. Sun · 2017
Later among the works it cites.
Thinet: A filter level pruning method for deep neural network compression
J.-H. Luo, J. Wu, and W. Lin · 2017
Later among the works it cites.
Coordinating filters for faster deep neural networks
W. Wen, C. Xu, C. Wu, Y. Wang, Y. Chen, and H. Li · 2017
Later among the works it cites.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2018
Closest in time.
Network decoupling: From regular to depthwise separable convolutions
J. Guo, Y. Li, W. Lin, Y. Chen, and J. Li · 2018
Closest in time.
Thinet: pruning cnn filters for a thinner net
J.-H. Luo, H. Zhang, H.-Y. Zhou, C.-W. Xie, J. Wu, and W. Lin · 2018
Closest in time.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…