Fetching the paper…
Reading the bibliography…
The state of neural network pruning has been noticed to be unclear and even confusing for a while, largely due to "a lack of standardized benchmarks and metrics" [3].
What size net gives valid generalization?
Eric Baum and David Haussler · 1988
Earlier work this paper cites.
A back-propagation algorithm with optimal use of hidden units
Yves Chauvin · 1988
Earlier work this paper cites.
Skeletonization: A technique for trimming the fat from a network via relevance assessment
Michael C Mozer and Paul Smolensky · 1988
Earlier work this paper cites.
Optimal brain damage
Y. LeCun, J. S. Denker, and S. A. Solla · 1990
Earlier work this paper cites.
Second order derivatives for network pruning: Optimal brain surgeon
B. Hassibi and D. G. Stork · 1993
Earlier work this paper cites.
Pruning algorithms – a survey
R. Reed · 1993
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky · 2009
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio · 2010
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
On the importance of initialization and momentum in deep learning
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton · 2013
Earlier work this paper cites.
Caffe: Convolutional architecture for fast feature embedding
Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell · 2014
Earlier work this paper cites.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Andrew M Saxe, James L McClelland, and Surya Ganguli · 2014
Earlier work this paper cites.
Learning both weights and connections for efficient neural network
Song Han, Jeff Pool, John Tran, and William J Dally · 2015
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Earlier work this paper cites.
Deep learning
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton · 2015
Earlier work this paper cites.
Deep learning in neural networks: An overview
Jürgen Schmidhuber · 2015
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2015
Earlier work this paper cites.
Deep unsupervised learning using nonequilibrium thermodynamics
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli · 2015
Earlier work this paper cites.
TensorFlow: A system for large-scale machine learning
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al · 2016
Earlier work this paper cites.
Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding
Song Han, Huizi Mao, and William J Dally · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
Data-dependent initializations of convolutional neural networks
Philipp Krähenbühl, Carl Doersch, Jeff Donahue, and Trevor Darrell · 2016
Earlier work this paper cites.
All you need is a good init
Dmytro Mishkin and Jiri Matas · 2016
Earlier work this paper cites.
Learning structured sparsity in deep neural networks
Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li · 2016
Earlier work this paper cites.
Channel pruning for accelerating very deep neural networks
Yihui He, Xiangyu Zhang, and Jian Sun · 2017
Earlier work this paper cites.
Densely connected convolutional networks
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger · 2017
Earlier work this paper cites.
Pruning filters for efficient convnets
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf · 2017
Earlier work this paper cites.
Learning efficient convolutional networks through network slimming
Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang · 2017
Earlier work this paper cites.
Sgdr: Stochastic gradient descent with warm restarts
Ilya Loshchilov and Frank Hutter · 2017
Earlier work this paper cites.
Thinet: A filter level pruning method for deep neural network compression
Jian-Hao Luo, Jianxin Wu, and Weiyao Lin · 2017
Earlier work this paper cites.
Exploring the granularity of sparsity in convolutional neural networks
Huizi Mao, Song Han, Jeff Pool, Wenshuo Li, Xingyu Liu, Yu Wang, and William J Dally · 2017
Cited alongside, same era.
Pruning convolutional neural networks for resource efficient inference
P. Molchanov, S. Tyree, and T. Karras · 2017
Cited alongside, same era.
Efficient processing of deep neural networks: A tutorial and survey
Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Neural architecture search with reinforcement learning
Barret Zoph and Quoc Le · 2017
Cited alongside, same era.
Recent advances in efficient computation of deep convolutional neural networks
Jian Cheng, Pei-song Wang, Gang Li, Qing-hao Hu, and Han-qing Lu · 2018
What is the state of neural network pruning?
Davis Blalock, Jose Javier Gonzalez, Jonathan Frankle, and John V Guttag · 2020
Later among the works it cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Later among the works it cites.
Model compression and hardware acceleration for neural networks: A comprehensive survey
Lei Deng, Guoqi Li, Song Han, Luping Shi, and Yuan Xie · 2020
Later among the works it cites.
Rigging the lottery: Making all tickets winners
Utku Evci, Trevor Gale, Jacob Menick, Pablo Samuel Castro, and Erich Elsen · 2020
Later among the works it cites.
Linear mode connectivity and the lottery ticket hypothesis
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin · 2020
Later among the works it cites.
Learning filter pruning criteria for deep convolutional neural networks acceleration
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Model compression and acceleration for deep neural networks: The principles, progress, and challenges
Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang · 2018
Cited alongside, same era.
A closer look at structured pruning for neural network compression
Elliot J Crowley, Jack Turner, Amos Storkey, and Michael O’Boyle · 2018
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Cited alongside, same era.
Auto-balanced filter pruning for efficient convolutional neural networks
Xiaohan Ding, Guiguang Ding, Jungong Han, and Sheng Tang · 2018
Cited alongside, same era.
Soft filter pruning for accelerating deep convolutional neural networks
Yang He, Guoliang Kang, Xuanyi Dong, Yanwei Fu, and Yi Yang · 2018
Cited alongside, same era.
Squeeze-and-excitation networks
Jie Hu, Li Shen, and Gang Sun · 2018
Cited alongside, same era.
Yang He, Yuhang Ding, Ping Liu, Linchao Zhu, Hanwang Zhang, and Yi Yang · 2020
Later among the works it cites.
A signal propagation perspective for pruning neural networks at initialization
Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould, and Philip HS Torr · 2020
Later among the works it cites.
Provable filter pruning for efficient neural networks
Lucas Liebenwein, Cenk Baykal, Harry Lang, Dan Feldman, and Daniela Rus · 2020
Later among the works it cites.
Hrank: Filter pruning using high-rank feature map
Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, and Ling Shao · 2020
Later among the works it cites.
What’s hidden in a randomly weighted neural network?
Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari · 2020
Later among the works it cites.
Comparing rewinding and fine-tuning in neural network pruning
Alex Renda, Jonathan Frankle, and Michael Carbin · 2020
Later among the works it cites.
Woodfisher: Efficient second-order approximations for model compression
Sidak Pal Singh and Dan Alistarh · 2020
Later among the works it cites.
Picking winning tickets before training by preserving gradient flow
Chaoqi Wang, Guodong Zhang, and Roger Grosse · 2020
Later among the works it cites.
On the opportunities and risks of foundation models
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al · 2021
Later among the works it cites.
Resrep: Lossless cnn pruning via decoupling remembering and forgetting
Xiaohan Ding, Tianxiang Hao, Jianchao Tan, Ji Liu, Jungong Han, Yuchen Guo, and Guiguang Ding · 2021
Later among the works it cites.
Pruning neural networks at initialization: Why are we missing the mark?
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M Roy, and Michael Carbin · 2021
Later among the works it cites.
Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste · 2021
Later among the works it cites.
Network pruning that matters: A case study on retraining variants
Duong H Le and Binh-Son Hua · 2021
Later among the works it cites.
Towards compact cnns via collaborative compression
Yuchao Li, Shaohui Lin, Jianzhuang Liu, Qixiang Ye, Mengdi Wang, Fei Chao, Fan Yang, Jincheng Ma, Qi Tian, and Rongrong Ji · 2021
Later among the works it cites.
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021
Later among the works it cites.
Neural pruning via growing regularization
Huan Wang, Can Qin, Yulun Zhang, and Yun Fu · 2021
Later among the works it cites.
Aligned structured sparsity learning for efficient image super-resolution
Yulun Zhang, Huan Wang, Can Qin, and Yun Fu · 2021
Later among the works it cites.
Learning n: M fine-grained structured sparse neural networks from scratch
Aojun Zhou, Yukun Ma, Junnan Zhu, Jianbo Liu, Zhijie Zhang, Kun Yuan, Wenxiu Sun, and Hongsheng Li · 2021
Later among the works it cites.
Attentive fine-grained structured sparsity for image restoration
Junghun Oh, Heewon Kim, Seungjun Nah, Cheeun Hong, Jonghyun Choi, and Kyoung Mu Lee · 2022
Later among the works it cites.
ChatGPT: Optimizing Language Models for Dialogue
OpenAI · 2022
Later among the works it cites.
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer · 2022
Later among the works it cites.
Trainability preserving neural structured pruning
Huan Wang and Yun Fu · 2022
Later among the works it cites.
Recent advances on neural network pruning at initialization
Huan Wang, Can Qin, Yulun Zhang, and Yun Fu · 2022
Later among the works it cites.
Learning efficient image super-resolution networks via structure-regularized pruning
Yulun Zhang, Huan Wang, Can Qin, and Yun Fu · 2022
Later among the works it cites.
Depgraph: Towards any structural pruning
Gongfan Fang, Xinyin Ma, Mingli Song, Michael Bi Mi, and Xinchao Wang · 2023
Closest in time.