Fetching the paper…
Reading the bibliography…
Large neural networks can be pruned to a small fraction of their original size, with little loss in accuracy, by following a time-consuming "train, prune, re-train" approach.
Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment
Mozer, M. C. and Smolensky, P · 1989
Earlier work this paper cites.
Optimal Brain Damage
LeCun, Y., Denker, J. S., and Solla, S. A · 1990
Earlier work this paper cites.
Second order derivatives for network pruning: Optimal Brain Surgeon
Hassibi, B. and Stork, D. G · 1993
Earlier work this paper cites.
Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories
Fei-Fei, L., Fergus, R., and Perona, P · 2004
Earlier work this paper cites.
Pruning neural networks at initialization: Why are we missing the mark?
Frankle, J., Dziugaite, G. K., Roy, D. M., and Carbin, M · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A. et al · 2009
Earlier work this paper cites.
Estimating or propagating gradients through stochastic neurons for conditional computation
Bengio, Y., Léonard, N., and Courville, A · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Simonyan, K. and Zisserman, A · 2014
Earlier work this paper cites.
Binaryconnect: Training deep neural networks with binary weights during propagations
Courbariaux, M., Bengio, Y., and David, J.-P · 2015
Earlier work this paper cites.
Learning both weights and connections for efficient neural network
Han, S., Pool, J., Tran, J., and Dally, W · 2015
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
He, K., Zhang, X., Ren, S., and Sun, J · 2015
Earlier work this paper cites.
Han, S., Mao, H., and Dally, W. J · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
Binarized neural networks
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., and Bengio, Y · 2016
Cited alongside, same era.
Zagoruyko, S. and Komodakis, N · 2016
Cited alongside, same era.
Quantized neural networks: Training neural networks with low precision weights and activations
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., and Bengio, Y · 2017
Cited alongside, same era.
To prune, or not to prune: exploring the efficacy of pruning for model compression
Zhu, M. and Gupta, S · 2017
Cited alongside, same era.
The lottery ticket hypothesis: Finding sparse, trainable neural networks
Frankle, J. and Carbin, M · 2018
Cited alongside, same era.
Proving the lottery ticket hypothesis: Pruning is all you need
Malach, E., Yehudai, G., Shalev-Schwartz, S., and Shamir, O · 2020
Later among the works it cites.
Logarithmic pruning is all you need
Orseau, L., Hutter, M., and Rivasplata, O · 2020
Later among the works it cites.
Optimal lottery tickets via subsetsum: Logarithmic over-parameterization is sufficient
Pensia, A., Rajput, S., Nagle, A., Vishwakarma, H., and Papailiopoulos, D · 2020
Later among the works it cites.
What’s hidden in a randomly weighted neural network?
Ramanujan, V., Wortsman, M., Kembhavi, A., Farhadi, A., and Rastegari, M · 2020
Later among the works it cites.
Comparing rewinding and fine-tuning in neural network pruning
Renda, A., Frankle, J., and Carbin, M · 2020
Later among the works it cites.
Sanity-checking pruning methods: Random tickets can win the jackpot
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Lee, N., Ajanthan, T., and Torr, P. H · 2018
Cited alongside, same era.
Mobilenetv2: Inverted residuals and linear bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C · 2018
Cited alongside, same era.
The state of sparsity in deep neural networks
Gale, T., Elsen, E., and Hooker, S · 2019
Cited alongside, same era.
A review of binarized neural networks
Simons, T. and Lee, D.-J · 2019
Cited alongside, same era.
Picking winning tickets before training by preserving gradient flow
Wang, C., Zhang, G., and Grosse, R · 2019
Cited alongside, same era.
The lottery ticket hypothesis for pre-trained bert networks
Chen, T., Frankle, J., Chang, S., Liu, S., Zhang, Y., Wang, Z., and Carbin, M · 2020
Cited alongside, same era.
Multi-prize lottery ticket hypothesis: Finding accurate binary neural networks by pruning a randomly weighted network
Diffenderfer, J. and Kailkhura, B · 2020
Cited alongside, same era.
Su, J., Chen, Y., Cai, T., Wu, T., Gao, R., Wang, L., and Lee, J. D · 2020
Later among the works it cites.
Pruning neural networks without any data by iteratively conserving synaptic flow
Tanaka, H., Kunin, D., Yamins, D. L., and Ganguli, S · 2020
Later among the works it cites.
Prospect pruning: Finding trainable weights at initialization using meta-gradients
Alizadeh, M., Tailor, S. A., Zintgraf, L. M., van Amersfoort, J., Farquhar, S., Lane, N. D., and Gal, Y · 2021
Later among the works it cites.
A unified lottery ticket hypothesis for graph neural networks
Chen, T., Sui, Y., Chen, X., Zhang, A., and Wang, Z · 2021
Later among the works it cites.
Towards strong pruning for lottery tickets with non-zero biases
Fischer, J. and Burkholz, R · 2021
Later among the works it cites.
The lottery ticket hypothesis for object recognition
Girish, S., Maiya, S. R., Gupta, K., Chen, H., Davis, L. S., and Shrivastava, A · 2021
Later among the works it cites.
Sanity checks for lottery tickets: Does your winning ticket really win the jackpot?
Ma, X., Yuan, G., Shen, X., Chen, T., Chen, X., Chen, X., Liu, N., Qin, M., Liu, S., Wang, Z., et al · 2021
Later among the works it cites.
Finding everything within random binary networks
Sreenivasan, K., Rajput, S., Sohn, J.-y., and Papailiopoulos, D · 2021
Later among the works it cites.