Fetching the paper…
Reading the bibliography…
Practitioners frequently observe that pruning improves model generalization.
On the uniform convergence of relative frequencies of events to their probabilities
V. N. Vapnik and A. Y. Chervonenkis · 1971
Earlier work this paper cites.
Skeletonization: A technique for trimming the fat from a network via relevance assessment
M. C. Mozer and P. Smolensky · 1988
Earlier work this paper cites.
Optimal brain damage
Y. LeCun, J. Denker, and S. Solla · 1990
Earlier work this paper cites.
Optimal brain surgeon and general network pruning
B. Hassibi, D. Stork, and G. Wolff · 1993
Earlier work this paper cites.
Pruning recurrent neural networks for improved generalization performance
C. Giles and C. Omlin · 1994
Earlier work this paper cites.
Evaluating pruning methods
G. Thimm and E. Fiesler · 1995
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.
The nature of statistical learning theory
V. Vapnik · 1999
Earlier work this paper cites.
Occam's razor
C. Rasmussen and Z. Ghahramani · 2000
Earlier work this paper cites.
Rademacher and gaussian complexities: Risk bounds and structural results
P. L. Bartlett and S. Mendelson · 2003
Earlier work this paper cites.
The minimum description length principle
P. D. Grünwald · 2007
Earlier work this paper cites.
Building classifiers with independency constraints
T. Calders, F. Kamiran, and M. Pechenizkiy · 2009
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
A. Krizhevsky and G. Hinton · 2009
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
Pruning algorithms of neural networks — a comparative study
M. Augasta and T. Kathirvalavakumar · 2013
Earlier work this paper cites.
In search of the real inductive bias: On the role of implicit regularization in deep learning
B. Neyshabur, R. Tomioka, and N. Srebro · 2014
Earlier work this paper cites.
Learning both weights and connections for efficient neural networks
S. Han, J. Pool, J. Tran, and W. J. Dally · 2015
Earlier work this paper cites.
Norm-based capacity control in neural networks
B. Neyshabur, R. Tomioka, and N. Srebro · 2015
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2015
Earlier work this paper cites.
Going deeper with convolutions
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich · 2015
Earlier work this paper cites.
Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data
G. K. Dziugaite and D. M. Roy · 2016
Earlier work this paper cites.
Deep Learning
I. Goodfellow, Y. Bengio, and A. Courville · 2016
Cited alongside, same era.
Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding
S. Han, H. Mao, and W. J. Dally · 2016
Cited alongside, same era.
Equality of opportunity in supervised learning
M. Hardt, E. Price, E. Price, and N. Srebro · 2016
Cited alongside, same era.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Cited alongside, same era.
Fast convnets using group-wise brain damage
V. Lebedev and V. Lempitsky · 2016
Cited alongside, same era.
Learning to prune deep neural networks via layer-wise optimal brain surgeon
X. Dong, S. Chen, and S. Pan · 2017
Cited alongside, same era.
EigenDamage: Structured pruning in the Kronecker-factored eigenbasis
C. Wang, R. Grosse, S. Fidler, and G. Zhang · 2019
Later among the works it cites.
The generalization-stability tradeoff in neural network pruning
B. Bartoldson, A. Morcos, A. Barbu, and G. Erlebacher · 2020
Later among the works it cites.
What is the state of neural network pruning?
D. Blalock, J. J. Gonzalez Ortiz, J. Frankle, and J. Guttag · 2020
Later among the works it cites.
Linear mode connectivity and the lottery ticket hypothesis
J. Frankle, G. K. Dziugaite, D. M. Roy, and M. Carbin · 2020
Later among the works it cites.
Characterising bias in compressed models
S. Hooker, N. Moorosi, G. Clark, S. Bengio, and E. Denton · 2020
Later among the works it cites.
A signal propagation perspective for pruning neural networks at initialization
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2.9 Model Selection and Bias-Variance Tradeoff , page 37–38
T. Hastie, J. Friedman, and R. Tisbshirani · 2017
Cited alongside, same era.
Pruning filters for efficient convnets
H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf · 2017
Cited alongside, same era.
Learning efficient convolutional networks through network slimming
Z. Liu, J. Li, Z. Shen, G. Huang, S. Yan, and C. Zhang · 2017
Cited alongside, same era.
Cyclical learning rates for training neural networks
L. N. Smith · 2017
Cited alongside, same era.
NISP: pruning networks using neuron importance score propagation
R. Yu, A. Li, C. Chen, J. Lai, V. I. Morariu, X. Han, M. Gao, C. Lin, and L. S. Davis · 2017
Cited alongside, same era.
Learning sparse neural networks through l0 regularization
C. Louizos, M. Welling, and D. P. Kingma · 2018
Cited alongside, same era.
N. Lee, T. Ajanthan, S. Gould, and P. H. S. Torr · 2020
Later among the works it cites.
Provable filter pruning for efficient neural networks
L. Liebenwein, C. Baykal, H. Lang, D. Feldman, and D. Rus · 2020
Later among the works it cites.
Deep double descent: Where bigger models and more data hurt
P. Nakkiran, G. Kaplun, Y. Bansal, T. Yang, B. Barak, and I. Sutskever · 2020
Later among the works it cites.
Comparing rewinding and fine-tuning in neural network pruning
A. Renda, J. Frankle, and M. Carbin · 2020
Later among the works it cites.
Lossless compression of deep neural networks
T. Serra, A. Kumar, and S. Ramalingam · 2020
Later among the works it cites.
Woodfisher: Efficient second-order approximation for neural network compression
S. P. Singh and D. Alistarh · 2020
Later among the works it cites.
Pruning neural networks without any data by iteratively conserving synaptic flow
H. Tanaka, D. Kunin, D. L. Yamins, and S. Ganguli · 2020
Later among the works it cites.
Picking winning tickets before training by preserving gradient flow
C. Wang, G. Zhang, and R. Grosse · 2020
Later among the works it cites.
Simon says: Evaluating and mitigating bias in pruned neural networks with knowledge distillation
C. Blakeney, N. Huish, Y. Yan, and Z. Zong · 2021
Later among the works it cites.
Only train once: A one-shot neural network training and pruning framework
T. Chen, B. Ji, T. Ding, B. Fang, G. Wang, Z. Zhu, L. Liang, Y. Shi, S. Yi, and X. Tu · 2021
Later among the works it cites.
Pruning neural networks at initialization: Why are we missing the mark?
J. Frankle, G. K. Dziugaite, D. Roy, and M. Carbin · 2021
Later among the works it cites.
Layer-adaptive sparsity for the magnitude-based pruning
J. Lee, S. Park, S. Mo, S. Ahn, and J. Shin · 2021
Later among the works it cites.
Lost in pruning: The effects of pruning neural networks beyond test accuracy
L. Liebenwein, C. Baykal, B. Carter, D. Gifford, and D. Rus · 2021
Later among the works it cites.
Deep learning on a data diet: Finding important examples early in training
M. Paul, S. Ganguli, and G. K. Dziugaite · 2021
Later among the works it cites.
Scaling up exact neural network compression by relu stability
T. Serra, X. Yu, A. Kumar, and S. Ramalingam · 2021
Later among the works it cites.
Understanding deep learning (still) requires rethinking generalization
C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals · 2021
Later among the works it cites.
The combinatorial brain surgeon: Pruning weights that cancel one another in neural networks
X. Yu, T. Serra, S. Ramalingam, and S. Zhe · 2022
Closest in time.