Fetching the paper…
Reading the bibliography…
Works on lottery ticket hypothesis (LTH) and single-shot network pruning (SNIP) have raised a lot of attention currently on post-training pruning (iterative magnitude pruning), and before-training pruning (pruning at initialization).
A signal propagation perspective for pruning neural networks at initialization
N. Lee, T. Ajanthan, S. Gould, and P. H. S. Torr · 1906
Earlier work this paper cites.
Pruning versus clipping in neural networks
S. A. Janowsky · 1989
Earlier work this paper cites.
Skeletonization: A technique for trimming the fat from a network via relevance assessment
M. C. Mozer and P. Smolensky · 1989
Earlier work this paper cites.
Using relevance to reduce network size automatically
M. C. Mozer and P. Smolensky · 1989
Earlier work this paper cites.
Optimal brain damage
Y. LeCun, J. S. Denker, and S. A. Solla · 1990
Earlier work this paper cites.
Second order derivatives for network pruning: Optimal brain surgeon
B. Hassibi and D. G. Stork · 1993
Earlier work this paper cites.
Principles of neural science
E. R. Kandel, J. H. Schwartz, T. M. Jessell, S. Siegelbaum, A. J. Hudspeth, and S. Mack · 2000
Earlier work this paper cites.
Comparing rewinding and fine-tuning in neural network pruning
A. Renda, J. Frankle, and M. Carbin · 2003
Earlier work this paper cites.
Progressive skeletonization: Trimming more fat from a network at initialization
P. de Jorge, A. Sanyal, H. Behl, P. Torr, G. Rogez, and P. K. Dokania · 2006
Earlier work this paper cites.
Glial inhibition of cns axon regeneration
G. Yiu and Z. He · 2006
Earlier work this paper cites.
Pruning neural networks at initialization: Why are we missing the mark?
J. Frankle, G. K. Dziugaite, D. Roy, and M. Carbin · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
A. Krizhevsky and G. Hinton · 2009
Earlier work this paper cites.
Exploiting linear structure within convolutional networks for efficient evaluation
E. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus · 2014
Earlier work this paper cites.
Speeding up convolutional neural networks with low rank expansions
M. Jaderberg, A. Vedaldi, and A. Zisserman · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2014
Earlier work this paper cites.
Learning both weights and connections for efficient neural network
S. Han, J. Pool, J. Tran, and W. Dally · 2015
Earlier work this paper cites.
Scalable distributed dnn training using commodity gpu cloud computing
N. Strom · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
Pruning filters for efficient convnets
H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf · 2016
Earlier work this paper cites.
Pruning convolutional neural networks for resource efficient inference
P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz · 2016
Earlier work this paper cites.
Learning structured sparsity in deep neural networks
W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li · 2016
Earlier work this paper cites.
Deep rewiring: Training very sparse deep networks
G. Bellec, D. Kappel, W. Maass, and R. Legenstein · 2017
Earlier work this paper cites.
Variational dropout sparsifies deep neural networks
D. Molchanov, A. Ashukha, and D. Vetrov · 2017
Earlier work this paper cites.
Training sparse neural networks
S. Srinivas, A. Subramanya, and R. Venkatesh Babu · 2017
Earlier work this paper cites.
Network decoupling: From regular to depthwise separable convolutions
J. Guo, Y. Li, W. Lin, Y. Chen, and J. Li · 2018
Earlier work this paper cites.
Snip: Single-shot network pruning based on connection sensitivity
N. Lee, T. Ajanthan, and P. H. Torr · 2018
Earlier work this paper cites.
Learning sparse neural networks through l _ 0 l\_0 regularization
C. Louizos, M. Welling, and D. P. Kingma · 2018
Cited alongside, same era.
Intrinsic mechanisms of neuronal axon regeneration
M. Mahar and V. Cavalli · 2018
Cited alongside, same era.
Recovering from random pruning: On the plasticity of deep convolutional neural networks
D. Mittal, S. Bhardwaj, M. M. Khapra, and B. Ravindran · 2018
Cited alongside, same era.
Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science
D. C. Mocanu, E. Mocanu, P. Stone, P. H. Nguyen, M. Gibescu, and A. Liotta · 2018
Cited alongside, same era.
Trained rank pruning for efficient deep neural networks
Y. Xu, Y. Li, S. Zhang, W. Wen, B. Wang, Y. Qi, Y. Chen, W. Lin, and H. Xiong · 2018
Cited alongside, same era.
Top-kast: Top-k always sparse training
S. Jayakumar, R. Pascanu, J. Rae, S. Osindero, and E. Elsen · 2020
Later among the works it cites.
Soft threshold weight reparameterization for learnable sparsity
A. Kusupati, V. Ramanujan, R. Somani, M. Wortsman, P. Jain, S. Kakade, and A. Farhadi · 2020
Later among the works it cites.
Dynamic model pruning with feedback
T. Lin, S. U. Stich, L. Barba, D. Dmitriev, and M. Jaggi · 2020
Later among the works it cites.
Dynamic sparse training: Find efficient sparse network from scratch with trainable masked layers
J. Liu, Z. Xu, R. Shi, R. C. C. Cheung, and H. K. So · 2020
Later among the works it cites.
Finding trainable sparse networks through neural tangent transfer
T. Liu and F. Zenke · 2020
Later among the works it cites.
Neuroregeneration and plasticity: a review of the physiological mechanisms for achieving functional recovery postinjury
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
To prune, or not to prune: Exploring the efficacy of pruning for model compression, 2018
M. H. Zhu and S. Gupta · 2018
Cited alongside, same era.
Sparse networks from scratch: Faster training without losing performance
T. Dettmers and L. Zettlemoyer · 2019
Cited alongside, same era.
The lottery ticket hypothesis: Finding sparse, trainable neural networks
J. Frankle and M. Carbin · 2019
Cited alongside, same era.
The lottery ticket hypothesis at scale
J. Frankle, G. K. Dziugaite, D. M. Roy, and M. Carbin · 2019
Cited alongside, same era.
The state of sparsity in deep neural networks
T. Gale, E. Elsen, and S. Hooker · 2019
Cited alongside, same era.
Estimation of energy consumption in machine learning
E. García-Martín, C. F. Rodrigues, G. Riley, and H. Grahn · 2019
Cited alongside, same era.
Rethinking the value of network pruning
Z. Liu, M. Sun, T. Zhou, G. Huang, and T. Darrell · 2019
Cited alongside, same era.
P. G. Nagappan, H. Chen, and D.-Y. Wang · 2020
Later among the works it cites.
Nvidia a100 tensor core gpu architecture
Nvidia · 2020
Later among the works it cites.
Cpot: Channel pruning via optimal transport
Y. Shen, L. Shen, H.-Z. Huang, X. Wang, and W. Liu · 2020
Later among the works it cites.
Woodfisher: Efficient second-order approximation for neural network compression
S. P. Singh and D. Alistarh · 2020
Later among the works it cites.
Pruning neural networks without any data by iteratively conserving synaptic flow
H. Tanaka, D. Kunin, D. L. Yamins, and S. Ganguli · 2020
Later among the works it cites.
Pruning via iterative ranking of sensitivity statistics
S. Verdenius, M. Stol, and P. Forré · 2020
Later among the works it cites.
Picking winning tickets before training by preserving gradient flow
C. Wang, G. Zhang, and R. Grosse · 2020
Later among the works it cites.
Procrustes: a dataflow and accelerator for sparse deep neural network training
D. Yang, A. Ghasemazar, X. Ren, M. Golub, G. Lemieux, and M. Lis · 2020
Later among the works it cites.
A block decomposition algorithm for sparse optimization
G. Yuan, L. Shen, and W.-S. Zheng · 2020
Later among the works it cites.
The lottery tickets hypothesis for supervised and self-supervised pre-training in computer vision models
T. Chen, J. Frankle, S. Chang, S. Liu, Y. Zhang, M. Carbin, and Z. Wang · 2021
Closest in time.
A unified paths perspective for pruning at initialization
T. Gebhart, U. Saxena, and P. Schrater · 2021
Closest in time.
Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks
T. Hoefler, D. Alistarh, T. Ben-Nun, N. Dryden, and A. Peste · 2021
Closest in time.
Accelerated sparse neural training: A provable and efficient method to find n: M transposable masks
I. Hubara, B. Chmiel, M. Island, R. Banner, S. Naor, and D. Soudry · 2021
Closest in time.
Sparse evolutionary deep learning with over one million artificial neurons on commodity hardware
S. Liu, D. C. Mocanu, A. R. R. Matavalam, Y. Pei, and M. Pechenizkiy · 2021
Closest in time.
Selfish sparse rnn training
S. Liu, D. C. Mocanu, Y. Pei, and M. Pechenizkiy · 2021
Closest in time.
Do we actually need dense over-parameterization? in-time over-parameterization in sparse training
S. Liu, L. Yin, D. C. Mocanu, and M. Pechenizkiy · 2021
Closest in time.
Sparse training theory for scalable and efficient agents
D. C. Mocanu, E. Mocanu, T. Pinto, S. Curci, P. H. Nguyen, M. Gibescu, D. Ernst, and Z. A. Vale · 2021
Closest in time.
Carbon emissions and large neural network training
D. Patterson, J. Gonzalez, Q. Le, C. Liang, L.-M. Munguia, D. Rothchild, D. So, M. Texier, and J. Dean · 2021
Closest in time.
Sparsifying networks via subdifferential inclusion
S. Verma and J.-C. Pesquet · 2021
Closest in time.
Learning n:m fine-grained structured sparse neural networks from scratch
A. Zhou, Y. Ma, J. Zhu, J. Liu, Z. Zhang, K. Yuan, W. Sun, and H. Li · 2021
Closest in time.