Fetching the paper…
Reading the bibliography…
Recently, sparse training has emerged as a promising paradigm for efficient deep learning on edge devices.
Scaling learning algorithms towards ai
Yoshua Bengio, Yann LeCun, et al · 2007
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky and Geoffrey Hinton · 2009
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding
Song Han, Huizi Mao, and William J. Dally · 2016
Earlier work this paper cites.
Dynamic network surgery for efficient dnns
Yiwen Guo, Anbang Yao, and Yurong Chen · 2016
Earlier work this paper cites.
Network trimming: A data-driven neuron pruning approach towards efficient deep architectures
Hengyuan Hu, Rui Peng, Yu-Wing Tai, and Chi-Keung Tang · 2016
Earlier work this paper cites.
Learning structured sparsity in deep neural networks
Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li · 2016
Earlier work this paper cites.
Deep networks with stochastic depth
Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q Weinberger · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Freezeout: Accelerate training by progressively freezing layers
Andrew Brock, Theodore Lim, James M Ritchie, and Nick Weston · 2017
Earlier work this paper cites.
Yang Fan, Fei Tian, Tao Qin, Jiang Bian, and Tie-Yan Liu · 2017
Earlier work this paper cites.
Variational dropout sparsifies deep neural networks
Dmitry Molchanov, Arsenii Ashukha, and Dmitry Vetrov · 2017
Earlier work this paper cites.
Thinet: A filter level pruning method for deep neural network compression
Jian-Hao Luo, Jianxin Wu, and Weiyao Lin · 2017
Earlier work this paper cites.
Channel pruning for accelerating very deep neural networks
Yihui He, Xiangyu Zhang, and Jian Sun · 2017
Earlier work this paper cites.
Structured bayesian pruning via log-normal multiplicative noise
Kirill Neklyudov, Dmitry Molchanov, Arsenii Ashukha, and Dmitry Vetrov · 2017
Earlier work this paper cites.
An empirical study of example forgetting during deep neural network learning
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon · 2018
Earlier work this paper cites.
Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science
Decebal Constantin Mocanu, Elena Mocanu, Peter Stone, Phuong H Nguyen, Madeleine Gibescu, and Antonio Liotta · 2018
Earlier work this paper cites.
Nisp: Pruning networks using neuron importance score propagation
Ruichi Yu, Ang Li, Chun-Fu Chen, Jui-Hsin Lai, Vlad I Morariu, Xintong Han, Mingfei Gao, Ching-Yung Lin, and Larry S Davis · 2018
Earlier work this paper cites.
Soft filter pruning for accelerating deep convolutional neural networks
Yang He, Guoliang Kang, Xuanyi Dong, Yanwei Fu, and Yi Yang · 2018
Earlier work this paper cites.
Compressing neural networks using the variational information bottleneck
Bin Dai, Chen Zhu, Baining Guo, and David Wipf · 2018
Earlier work this paper cites.
Efficient hardware realization of convolutional neural networks using intra-kernel regular pruning
Maurice Yang, Mahmoud Faraj, Assem Hussein, and Vincent Gaudet · 2018
Earlier work this paper cites.
Deep rewiring: Training very sparse deep net-works
Guillaume Bellec, David Kappel, Wolfgang Maass, and Robert Legenstein · 2018
Earlier work this paper cites.
Snip: Single-shot network pruning based on connection sensitivity
Namhoon Lee, Thalaiyasingam Ajanthan, and Philip Torr · 2019
Cited alongside, same era.
Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization
Hesham Mostafa and Xin Wang · 2019
Cited alongside, same era.
Similarity of neural network representations revisited
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton · 2019
Cited alongside, same era.
Efficient training of bert by progressively stacking
Linyuan Gong, Di He, Zhuohan Li, Tao Qin, Liwei Wang, and Tieyan Liu · 2019
Cited alongside, same era.
What would elsa do? freezing layers during transformer fine-tuning
Jaejun Lee, Raphael Tang, and Jimmy Lin · 2019
Cited alongside, same era.
Network pruning via transformable architecture search
Drawing early-bird tickets: Toward more efficient training of deep networks
Haoran You, Chaojian Li, Pengfei Xu, Yonggan Fu, Yue Wang, Xiaohan Chen, Yingyan Lin, Zhangyang Wang, and Richard G. Baraniuk · 2020
Later among the works it cites.
Balancing training time vs. performance with bayesian early pruning
Mohit Rajpal, Yehong Zhang, Bryan Kian, and Hsiang Low · 2020
Later among the works it cites.
Pruning neural networks without any data by iteratively conserving synaptic flow
Hidenori Tanaka, Daniel Kunin, Daniel L Yamins, and Surya Ganguli · 2020
Later among the works it cites.
Freezenet: Full performance by reduced storage costs
Paul Wimmer, Jens Mehnert, and Alexandru Condurache · 2020
Later among the works it cites.
Single shot structured pruning before training
Joost van Amersfoort, Milad Alizadeh, Sebastian Farquhar, Nicholas Lane, and Yarin Gal · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Xuanyi Dong and Yi Yang · 2019
Cited alongside, same era.
Compressing convolutional neural networks via factorized convolutional filters
Tuanhui Li, Baoyuan Wu, Yujiu Yang, Yanbo Fan, Yong Zhang, and Wei Liu · 2019
Cited alongside, same era.
Filter pruning via geometric median for deep convolutional neural networks acceleration
Yang He, Ping Liu, Ziwei Wang, Zhilan Hu, and Yi Yang · 2019
Cited alongside, same era.
Prunetrain: fast neural network training by dynamic sparse model reconfiguration
Sangkug Lym, Esha Choukse, Siavash Zangeneh, Wei Wen, Sujay Sanghavi, and Mattan Erez · 2019
Cited alongside, same era.
Sparse networks from scratch: Faster training without losing performance
Tim Dettmers and Luke Zettlemoyer · 2019
Cited alongside, same era.
The lottery ticket hypothesis: Finding sparse, trainable neural networks
Jonathan Frankle and Michael Carbin · 2019
Cited alongside, same era.
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al · 2019
Cited alongside, same era.
Sheng Shen, Alexei Baevski, Ari S Morcos, Kurt Keutzer, Michael Auli, and Douwe Kiela · 2020
Later among the works it cites.
Why layer-wise learning is hard to scale-up and a possible solution via accelerated downsampling
Wenchi Ma, Miao Yu, Kaidong Li, and Guanghui Wang · 2020
Later among the works it cites.
Knowledge distillation from internal representations
Gustavo Aguilar, Yuan Ling, Yu Zhang, Benjamin Yao, Xing Fan, and Chenlei Guo · 2020
Later among the works it cites.
Mest: Accurate and fast memory-economic sparse training framework on the edge
Geng Yuan, Xiaolong Ma, Wei Niu, Zhengang Li, Zhenglun Kong, Ning Liu, Yifan Gong, Zheng Zhan, Chaoyang He, Qing Jin, et al · 2021
Later among the works it cites.
Do we actually need dense over-parameterization? in-time over-parameterization in sparse training
Shiwei Liu, Lu Yin, Decebal Constantin Mocanu, and Mykola Pechenizkiy · 2021
Later among the works it cites.
Autofreeze: Automatically freezing model blocks to accelerate fine-tuning
Yuhan Liu, Saurabh Agarwal, and Shivaram Venkataraman · 2021
Later among the works it cites.
Pipetransformer: Automated elastic pipelining for distributed training of transformers
Chaoyang He, Shen Li, Mahdi Soltanolkotabi, and Salman Avestimehr · 2021
Later among the works it cites.
Lottery ticket preserves weight correlation: Is it desirable or not?
Ning Liu, Geng Yuan, Zhengping Che, Xuan Shen, Xiaolong Ma, Qing Jin, Jian Ren, Jian Tang, Sijia Liu, and Yanzhi Wang · 2021
Later among the works it cites.
Structadmm: Achieving ultrahigh efficiency in structured pruning for dnns
Tianyun Zhang, Shaokai Ye, Xiaoyu Feng, Xiaolong Ma, Kaiqi Zhang, Zhengang Li, Jian Tang, Sijia Liu, Xue Lin, Yongpan Liu, et al · 2021
Later among the works it cites.
Non-structured dnn weight pruning–is it beneficial in any platform?
Xiaolong Ma, Sheng Lin, Shaokai Ye, Zhezhi He, Linfeng Zhang, Geng Yuan, Sia Huat Tan, Zhengang Li, Deliang Fan, Xuehai Qian, et al · 2021
Later among the works it cites.
Teachers do more than teach: Compressing image-to-image models
Qing Jin, Jian Ren, Oliver J Woodford, Jiazhuo Wang, Geng Yuan, Yanzhi Wang, and Sergey Tulyakov · 2021
Later among the works it cites.
Clicktrain: Efficient and accurate end-to-end deep learning training via fine-grained architecture-preserving pruning
Chengming Zhang, Geng Yuan, Wei Niu, Jiannan Tian, Sian Jin, Donglin Zhuang, Zhe Jiang, Yanzhi Wang, Bin Ren, Shuaiwen Leon Song, and Dingwen Tao · 2021
Later among the works it cites.
Grim: A general, real-time deep learning inference framework for mobile devices based on fine-grained structured weight sparsity
Wei Niu, Zhengang Li, Xiaolong Ma, Peiyan Dong, Gang Zhou, Xuehai Qian, Xue Lin, Yanzhi Wang, and Bin Ren · 2021
Later among the works it cites.
Cocopie: enabling real-time ai on off-the-shelf mobile devices via compression-compilation co-design
Hui Guan, Shaoshan Liu, Xiaolong Ma, Wei Niu, Bin Ren, Xipeng Shen, Yanzhi Wang, and Pu Zhao · 2021
Later among the works it cites.
Weight update skipping: Reducing training time for artificial neural networks
Pooneh Safayenikoo and Ismail Akturk · 2021
Later among the works it cites.
Efficient dnn training with knowledge-guided layer freezing
Yiding Wang, Decang Sun, Kai Chen, Fan Lai, and Mosharaf Chowdhury · 2022
Closest in time.