Pruning Randomly Initialized Neural Networks with Iterative Randomization

Learning representations by back-propagating errors

David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams · 1986

Earlier work this paper cites.

Comparing biases for minimal network construction with back-propagation

Stephen Hanson and Lorien Pratt · 1988

Earlier work this paper cites.

Optimal brain damage

Yann LeCun, John S Denker, and Sara A Solla · 1990

Earlier work this paper cites.

Learning multiple layers of features from tiny images

Alex Krizhevsky · 2009

Earlier work this paper cites.

Learning word vectors for sentiment analysis

Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts · 2011

Earlier work this paper cites.

Learning both weights and connections for efficient neural networks

Song Han, Jeff Pool, John Tran, and William J Dally · 2015

Earlier work this paper cites.

Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015

Earlier work this paper cites.

ImageNet Large Scale Visual Recognition Challenge

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei · 2015

Earlier work this paper cites.

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016

Earlier work this paper cites.

Wide residual networks

Sergey Zagoruyko and Nikos Komodakis · 2016

Earlier work this paper cites.

Dsd: Dense-sparse-dense training for deep neural networks

Song Han, Jeff Pool, Sharan Narang, Huizi Mao, Enhao Gong, Shijian Tang, Erich Elsen, Peter Vajda, Manohar Paluri, John Tran, et al · 2017

Earlier work this paper cites.

Sgdr: Stochastic gradient descent with warm restarts

Ilya Loshchilov and Frank Hutter · 2017

Earlier work this paper cites.

Variational dropout sparsifies deep neural networks

Dmitry Molchanov, Arsenii Ashukha, and Dmitry Vetrov · 2017

Cited alongside, same era.

Deep rewiring: Training very sparse deep networks

Guillaume Bellec, David Kappel, Wolfgang Maass, and Robert Legenstein · 2018

Cited alongside, same era.

Snip: Single-shot network pruning based on connection sensitivity

Namhoon Lee, Thalaiyasingam Ajanthan, and Philip Torr · 2018

Cited alongside, same era.

Learning sparse neural networks through l_0 regularization

Christos Louizos, Max Welling, and Diederik P Kingma · 2018

Cited alongside, same era.

Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science

Decebal Constantin Mocanu, Elena Mocanu, Peter Stone, Phuong H Nguyen, Madeleine Gibescu, and Antonio Liotta · 2018

Cited alongside, same era.

The lottery ticket hypothesis: Finding sparse, trainable neural networks

Logarithmic pruning is all you need

Laurent Orseau, Marcus Hutter, and Omar Rivasplata · 2020

Later among the works it cites.

Optimal lottery tickets via subset sum: Logarithmic over-parameterization is sufficient

Ankit Pensia, Shashank Rajput, Alliot Nagle, Harit Vishwakarma, and Dimitris Papailiopoulos · 2020

Later among the works it cites.

What’s hidden in a randomly weighted neural network?

Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari · 2020

Later among the works it cites.

Pruning neural networks without any data by iteratively conserving synaptic flow

Hidenori Tanaka, Daniel Kunin, Daniel L Yamins, and Surya Ganguli · 2020

Later among the works it cites.

Picking winning tickets before training by preserving gradient flow

Chaoqi Wang, Guodong Zhang, and Roger Grosse · 2020

Later among the works it cites.

Pruning from scratch

Yulong Wang, Xiaolu Zhang, Lingxi Xie, Jun Zhou, Hang Su, Bo Zhang, and Xiaolin Hu · 2020