Fetching the paper…
Reading the bibliography…
Compressed Sensing using $\ell_1$ regularization is among the most powerful and popular sparsification technique in many applications, but why has it not been used to obtain sparse deep learning model such as convolutional neural network (CNN)? This paper is aimed to provide an answer to this question and to show how to make it work.
Splitting algorithms for the sum of two nonlinear operators
P. L. Lions and B. Mercier · 1979
Earlier work this paper cites.
A minimization method for the sum of a convex function and a continuously differentiable function
Hisashi Mine and Masao Fukushima · 1981
Earlier work this paper cites.
Problem complexity and method efficiency in optimization
Arkadii Semenovich Nemirovsky and David Borisovich Yudin · 1983
Earlier work this paper cites.
Comparing biases for minimal network construction with back-propagation
Lorien Y. Pratt · 1988
Earlier work this paper cites.
Optimal brain damage
Yann LeCun, John S Denker, and Sara A Solla · 1990
Earlier work this paper cites.
Second order derivatives for network pruning: Optimal brain surgeon
Babak Hassibi and David G Stork · 1993
Earlier work this paper cites.
Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information
Emmanuel J Candès, Justin Romberg, and Terence Tao · 2006
Earlier work this paper cites.
Compressed sensing
D. L Donoho · 2006
Earlier work this paper cites.
Sparse mri: The application of compressed sensing for rapid mr imaging
Michael Lustig, David Donoho, and John M Pauly · 2007
Earlier work this paper cites.
Efficient online and batch learning using forward backward splitting
John Duchi and Yoram Singer · 2009
Earlier work this paper cites.
Sparse online learning via truncated gradient
John Langford, Lihong Li, and Tong Zhang · 2009
Earlier work this paper cites.
Primal-dual subgradient methods for convex problems
Yurii Nesterov · 2009
Earlier work this paper cites.
Dual averaging method for regularized stochastic learning and online optimization
Lin Xiao · 2009
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio · 2010
Cited alongside, same era.
Dual averaging methods for regularized stochastic learning and online optimization
Lin Xiao · 2010
Cited alongside, same era.
Incremental proximal methods for large scale convex optimization
Dimitri P. Bertsekas · 2011
Cited alongside, same era.
Follow-the-regularized-leader and mirror descent: Equivalence theorems and l1 regularization
Brendan McMahan · 2011
Cited alongside, same era.
Compressed sensing: theory and applications
Yonina C Eldar and Gitta Kutyniok · 2012
Cited alongside, same era.
Efficient backprop
Yann A LeCun, Léon Bottou, Genevieve B Orr, and Klaus-Robert Müller · 2012
Cited alongside, same era.
Network trimming: A data-driven neuron pruning approach towards efficient deep architectures
Hengyuan Hu, Rui Peng, Yu-Wing Tai, and Chi-Keung Tang · 2016
Later among the works it cites.
Pruning filters for efficient convnets
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf · 2016
Later among the works it cites.
Learning structured sparsity in deep neural networks
Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li · 2016
Later among the works it cites.
A survey of model compression and acceleration for deep neural networks
Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang · 2017
Later among the works it cites.
Channel pruning for accelerating very deep neural networks
Yihui He, Xiangyu Zhang, and Jian Sun · 2017
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Understanding the exploding gradient problem
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio · 2012
Cited alongside, same era.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2014
Cited alongside, same era.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Cited alongside, same era.
Deep learning
Y Lecun, Y Bengio, and G Hinton · 2015
Cited alongside, same era.
Learning the number of neurons in deep networks
Jose M Alvarez and Mathieu Salzmann · 2016
Cited alongside, same era.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Cited alongside, same era.
Data-driven sparse structure selection for deep neural networks
Zehao Huang and Naiyan Wang · 2017
Later among the works it cites.
Learning efficient convolutional networks through network slimming
Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang · 2017
Later among the works it cites.
Thinet: A filter level pruning method for deep neural network compression
Jian-Hao Luo, Jianxin Wu, and Weiyao Lin · 2017
Later among the works it cites.
A survey of algorithms and analysis for adaptive online learning
H Brendan McMahan · 2017
Later among the works it cites.
To prune, or not to prune: exploring the efficacy of pruning for model compression
Michael Zhu and Suyog Gupta · 2017
Later among the works it cites.
Rethinking the value of network pruning
Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell · 2018
Closest in time.
Recovering from random pruning: On the plasticity of deep convolutional neural networks
Deepak Mittal, Shweta Bhardwaj, Mitesh M. Khapra, and Balaraman Ravindran · 2018
Closest in time.