Fetching the paper…
Reading the bibliography…
The lottery ticket hypothesis (LTH) has shown that dense models contain highly sparse subnetworks (i.e., winning tickets) that can be trained in isolation to match full accuracy.
Comparing biases for minimal network construction with back-propagation
Hanson, S. and Pratt, L · 1988
Earlier work this paper cites.
Optimal brain damage
LeCun, Y., Denker, J. S., and Solla, S. A · 1990
Earlier work this paper cites.
The lottery ticket hypothesis for pre-trained bert networks
Chen, T., Frankle, J., Chang, S., Liu, S., Zhang, Y., Wang, Z., and Carbin, M · 2007
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G., et al · 2009
Earlier work this paper cites.
Chen, T., Frankle, J., Chang, S., Liu, S., Zhang, Y., Carbin, M., and Wang, Z · 2012
Earlier work this paper cites.
cudnn: Efficient primitives for deep learning
Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J., Catanzaro, B., and Shelhamer, E · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Simonyan, K. and Zisserman, A · 2014
Earlier work this paper cites.
Tiny imagenet visual recognition challenge
Le, Y. and Yang, X · 2015
Earlier work this paper cites.
Eie: efficient inference engine on compressed deep neural network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., and Dally, W. J · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
Network trimming: A data-driven neuron pruning approach towards efficient deep architectures
Hu, H., Peng, R., Tai, Y.-W., and Tang, C.-K · 2016
Earlier work this paper cites.
Pruning filters for efficient convnets
Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, H. P · 2016
Earlier work this paper cites.
Faster cnns with direct sparse convolutions and guided pruning
Park, J., Li, S., Wen, W., Tang, P. T. P., Li, H., Chen, Y., and Dubey, P · 2016
Earlier work this paper cites.
An overview of gradient descent optimization algorithms
Ruder, S · 2016
Earlier work this paper cites.
Learning structured sparsity in deep neural networks
Wen, W., Wu, C., Wang, Y., Chen, Y., and Li, H · 2016
Earlier work this paper cites.
Zagoruyko, S. and Komodakis, N · 2016
Earlier work this paper cites.
Channel pruning for accelerating very deep neural networks
He, Y., Zhang, X., and Sun, J · 2017
Earlier work this paper cites.
Mobilenets: Efficient convolutional neural networks for mobile vision applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H · 2017
Earlier work this paper cites.
Learning efficient convolutional networks through network slimming
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., and Zhang, C · 2017
Earlier work this paper cites.
Learning sparse neural networks through l _ 0 l\_0 regularization
Louizos, C., Welling, M., and Kingma, D. P · 2017
Cited alongside, same era.
Exploring the regularity of sparse structure in convolutional neural networks
Mao, H., Han, S., Pool, J., Li, W., Liu, X., Wang, Y., and Dally, W. J · 2017
Cited alongside, same era.
Adaptive runtime exploiting sparsity in tensor of deep learning neural network on heterogeneous systems
Peng, K.-Y., Fu, S.-Y., Liu, Y.-P., and Hsu, W.-C · 2017
Cited alongside, same era.
Sc-conv: Sparse-complementary convolution for efficient model utilization on cnns
Chen, C.-F., Oh, J., Fan, Q., and Pistoia, M · 2018
Cited alongside, same era.
Escort: Efficient sparse convolutional neural networks on gpus
Chen, X · 2018
Language models are few-shot learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Later among the works it cites.
A novel data transformation and execution strategy for accelerating sparse matrix multiplication on gpus
Jiang, P., Hong, C., and Agrawal, G · 2020
Later among the works it cites.
Scaling laws for neural language models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D · 2020
Later among the works it cites.
Channel pruning via automatic structure search
Lin, M., Ji, R., Zhang, Y., Zhang, B., Wu, Y., and Tian, Y · 2020
Later among the works it cites.
Pconv: The missing but desirable sparsity in dnn weight pruning for real-time execution on mobile devices
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Gradient descent provably optimizes over-parameterized neural networks
Du, S. S., Zhai, X., Poczos, B., and Singh, A · 2018
Cited alongside, same era.
Efficient sparse-matrix multi-vector product on gpus
Hong, C., Sukumaran-Rajam, A., Bandyopadhyay, B., Kim, J., Kurt, S. E., Nisa, I., Sabhlok, S., Çatalyürek, U. V., Parthasarathy, S., and Sadayappan, P · 2018
Cited alongside, same era.
Admm-nn: An algorithm-hardware co-design framework of dnns using alternating direction method of multipliers, 2018
Ren, A., Zhang, T., Ye, S., Li, J., Xu, W., Qian, X., Lin, X., and Wang, Y · 2018
Cited alongside, same era.
A systematic DNN weight pruning framework using alternating direction method of multipliers
Zhang, T., Ye, S., Zhang, K., Tang, J., Wen, W., Fardad, M., and Wang, Y · 2018
Cited alongside, same era.
The generalization-stability tradeoff in neural network pruning
Bartoldson, B. R., Morcos, A. S., Barbu, A., and Erlebacher, G · 2019
Cited alongside, same era.
Acorns: A framework for accelerating deep neural networks with input sparsity
Dong, X., Liu, L., Zhao, P., Li, G., Li, J., Wang, X., and Feng, X · 2019
Cited alongside, same era.
The lottery ticket hypothesis: Finding sparse, trainable neural networks
Frankle, J. and Carbin, M · 2019
Cited alongside, same era.
Ma, X., Guo, F.-M., Niu, W., Lin, X., Tang, J., Ma, K., Ren, B., and Wang, Y · 2020
Later among the works it cites.
Patdnn: Achieving real-time dnn execution on mobile devices with pattern-based weight pruning
Niu, W., Ma, X., Lin, S., Wang, S., Qian, X., Lin, X., Wang, Y., and Ren, B · 2020
Later among the works it cites.
Comparing rewinding and fine-tuning in neural network pruning
Renda, A., Frankle, J., and Carbin, M · 2020
Later among the works it cites.
Accelerating sparse cnn inference on gpus with performance-aware weight pruning
Rumi, M. A., Ma, X., Wang, Y., and Jiang, P · 2020
Later among the works it cites.
Pruning neural networks without any data by iteratively conserving synaptic flow
Tanaka, H., Kunin, D., Yamins, D. L., and Ganguli, S · 2020
Later among the works it cites.
Scop: Scientific control for reliable neural network pruning
Tang, Y., Wang, Y., Xu, Y., Tao, D., Xu, C., Xu, C., and Xu, C · 2020
Later among the works it cites.
Picking winning tickets before training by preserving gradient flow
Wang, C., Zhang, G., and Grosse, R · 2020
Later among the works it cites.
Drawing early-bird tickets: Toward more efficient training of deep networks
You, H., Li, C., Xu, P., Fu, Y., Wang, Y., Chen, X., Baraniuk, R. G., Wang, Z., and Lin, Y · 2020
Later among the works it cites.
Playing the lottery with rewards and multiple languages: lottery tickets in rl and nlp
Yu, H., Edunov, S., Tian, Y., and Morcos, A. S · 2020
Later among the works it cites.
A generalized lottery ticket hypothesis
Alabdulmohsin, I., Markeeva, L., Keysers, D., and Tolstikhin, I · 2021
Later among the works it cites.
Deep learning: a statistical viewpoint
Bartlett, P. L., Montanari, A., and Rakhlin, A · 2021
Later among the works it cites.
Playing lottery tickets with vision and language
Gan, Z., Chen, Y.-C., Li, L., Chen, T., Cheng, Y., Wang, S., and Liu, J · 2021
Later among the works it cites.
Hoefler, T., Alistarh, D., Ben-Nun, T., Dryden, N., and Peste, A · 2021
Later among the works it cites.
Efficient lottery ticket finding: Less data is more
Zhang, Z., Chen, X., Chen, T., and Wang, Z · 2021
Later among the works it cites.
Learning n: M fine-grained structured sparse neural networks from scratch
Zhou, A., Ma, Y., Zhu, J., Liu, J., Zhang, Z., Yuan, K., Sun, W., and Li, H · 2021
Later among the works it cites.