Fetching the paper…
Reading the bibliography…
Many recent works have shown trainability plays a central role in neural network pruning -- unattended broken trainability can lead to severe under-performance and unintentionally amplify the effect of retraining learning rate, resulting in biased (or even misinterpreted) benchmark results.
Optimal brain damage
Y. LeCun, J. S. Denker, and S. A. Solla · 1990
Earlier work this paper cites.
Second order derivatives for network pruning: Optimal brain surgeon
B. Hassibi and D. G. Stork · 1993
Earlier work this paper cites.
Pruning algorithms – a survey
R. Reed · 1993
Earlier work this paper cites.
Numerical linear algebra , volume 50
Lloyd N Trefethen and David Bau III · 1997
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al · 1998
Earlier work this paper cites.
How to generate random matrices from the classical compact groups
Francesco Mezzadri · 2006
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky · 2009
Earlier work this paper cites.
Rectified linear units improve restricted boltzmann machines
Vinod Nair and Geoffrey E Hinton · 2010
Earlier work this paper cites.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Andrew M Saxe, James L McClelland, and Surya Ganguli · 2014
Earlier work this paper cites.
Learning both weights and connections for efficient neural network
Song Han, Jeff Pool, John Tran, and William J Dally · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
Fast convnets using group-wise brain damage
V. Lebedev and V. Lempitsky · 2016
Earlier work this paper cites.
Learning structured sparsity in deep neural networks
Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li · 2016
Earlier work this paper cites.
ESE: Efficient speech recognition engine with sparse lstm on fpga
Song Han, Junlong Kang, Huizi Mao, Yiming Hu, Xin Li, Yubin Li, Dongliang Xie, Hong Luo, Song Yao, Yu Wang, et al · 2017
Earlier work this paper cites.
Channel pruning for accelerating very deep neural networks
Yihui He, Xiangyu Zhang, and Jian Sun · 2017
Earlier work this paper cites.
Pruning filters for efficient convnets
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf · 2017
Earlier work this paper cites.
Learning efficient convolutional networks through network slimming
Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang · 2017
Earlier work this paper cites.
Pruning convolutional neural networks for resource efficient inference
P. Molchanov, S. Tyree, and T. Karras · 2017
Earlier work this paper cites.
Efficient processing of deep neural networks: A tutorial and survey
Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer · 2017
Earlier work this paper cites.
All you need is beyond a good init: Exploring better solution for training extremely deep convolutional neural networks with orthonormality and modulation
Di Xie, Jiang Xiong, and Shiliang Pu · 2017
Cited alongside, same era.
Can we gain more from orthogonality regularizations in training deep networks?
Nitin Bansal, Xiaohan Chen, and Zhangyang Wang · 2018
Cited alongside, same era.
Model compression and acceleration for deep neural networks: The principles, progress, and challenges
Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang · 2018
Cited alongside, same era.
Auto-balanced filter pruning for efficient convolutional neural networks
Xiaohan Ding, Guiguang Ding, Jungong Han, and Sheng Tang · 2018
Cited alongside, same era.
Soft filter pruning for accelerating deep convolutional neural networks
Yang He, Guoliang Kang, Xuanyi Dong, Yanwei Fu, and Yi Yang · 2018
Cited alongside, same era.
A signal propagation perspective for pruning neural networks at initialization
Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould, and Philip HS Torr · 2020
Later among the works it cites.
Eagleeye: Fast sub-net evaluation for efficient neural network pruning
Bailin Li, Bowen Wu, Jiang Su, and Guangrun Wang · 2020
Later among the works it cites.
Provable filter pruning for efficient neural networks
Lucas Liebenwein, Cenk Baykal, Harry Lang, Dan Feldman, and Daniela Rus · 2020
Later among the works it cites.
Hrank: Filter pruning using high-rank feature map
Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, and Ling Shao · 2020
Later among the works it cites.
Comparing rewinding and fine-tuning in neural network pruning
Alex Renda, Jonathan Frankle, and Michael Carbin · 2020
Later among the works it cites.
Woodfisher: Efficient second-order approximations for model compression
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Orthogonal weight normalization: Solution to optimization over multiple dependent stiefel manifolds in deep neural networks
Lei Huang, Xianglong Liu, Bo Lang, Adams Yu, Yongliang Wang, and Bo Li · 2018
Cited alongside, same era.
Visualizing the loss landscape of neural nets
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein · 2018
Cited alongside, same era.
Learning sparse neural networks through l _ 0 l\_0 regularization
Christos Louizos, Max Welling, and Diederik P Kingma · 2018
Cited alongside, same era.
Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers
Jianbo Ye, Xin Lu, Zhe Lin, and James Z Wang · 2018
Cited alongside, same era.
Discrimination-aware channel pruning for deep neural networks
Zhuangwei Zhuang, Mingkui Tan, Bohan Zhuang, Jing Liu, Yong Guo, Qingyao Wu, Junzhou Huang, and Jinhui Zhu · 2018
Cited alongside, same era.
The state of sparsity in deep neural networks
Trevor Gale, Erich Elsen, and Sara Hooker · 2019
Cited alongside, same era.
Snip: Single-shot network pruning based on connection sensitivity
Namhoon Lee, Thalaiyasingam Ajanthan, and Philip Torr · 2019
Cited alongside, same era.
Sidak Pal Singh and Dan Alistarh · 2020
Later among the works it cites.
Orthogonal convolutional neural networks
Jiayun Wang, Yubei Chen, Rudrasis Chakraborty, and Stella X Yu · 2020
Later among the works it cites.
Deephoyer: Learning sparser neural network with differentiable scale-invariant sparsity measures
Huanrui Yang, Wei Wen, and Hai Li · 2020
Later among the works it cites.
Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste · 2021
Later among the works it cites.
Network pruning that matters: A case study on retraining variants
Duong H Le and Binh-Son Hua · 2021
Later among the works it cites.
Towards compact cnns via collaborative compression
Yuchao Li, Shaohui Lin, Jianzhuang Liu, Qixiang Ye, Mengdi Wang, Fei Chao, Fan Yang, Jincheng Ma, Qi Tian, and Rongrong Ji · 2021
Later among the works it cites.
A gradient flow framework for analyzing network pruning
Ekdeep Singh Lubana and Robert P Dick · 2021
Later among the works it cites.
Locally free weight sharing for network width search
Xiu Su, Shan You, Tao Huang, Fei Wang, Chen Qian, Changshui Zhang, and Chang Xu · 2021
Later among the works it cites.
Connectivity matters: Neural network pruning through the lens of effective sparsity
Artem Vysogorets and Julia Kempe · 2021
Later among the works it cites.
Dynamical isometry: The missing ingredient for neural network pruning
Huan Wang, Can Qin, Yue Bai, and Yun Fu · 2021
Later among the works it cites.
Resnet strikes back: An improved training procedure in timm
Ross Wightman, Hugo Touvron, and Hervé Jégou · 2021
Later among the works it cites.
Coarsening the granularity: Towards structurally sparse lottery tickets
Tianlong Chen, Xuxi Chen, Xiaolong Ma, Yanzhi Wang, and Zhangyang Wang · 2022
Closest in time.
Chex: Channel exploration for cnn model compression
Zejiang Hou, Minghai Qin, Fei Sun, Xiaolong Ma, Kun Yuan, Yi Xu, Yen-Kuang Chen, Rong Jin, Yuan Xie, and Sun-Yuan Kung · 2022
Closest in time.
Recent advances on neural network pruning at initialization
Huan Wang, Can Qin, Yulun Zhang, and Yun Fu · 2022
Closest in time.
Learning efficient image super-resolution networks via structure-regularized pruning
Yulun Zhang, Huan Wang, Can Qin, and Yun Fu · 2022
Closest in time.
Huan Wang, Can Qin, Yue Bai, and Yun Fu · 2023
Closest in time.