Fetching the paper…
Reading the bibliography…
The Lottery Ticket Hypothesis (LTH) states that a dense neural network model contains a highly sparse subnetwork (i.e., winning tickets) that can achieve even better performance than the original model when trained in isolation.
The lottery ticket hypothesis at scale
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M. Roy, and Michael Carbin · 1903
Earlier work this paper cites.
Sparse transfer learning via winning lottery tickets
Rahul Mehta · 1905
Earlier work this paper cites.
Optimal brain damage
Yann LeCun, John S. Denker, and Sara A. Solla · 1989
Earlier work this paper cites.
Picking winning tickets before training by preserving gradient flow
Chaoqi Wang, Guodong Zhang, and Roger Grosse · 2002
Earlier work this paper cites.
Lottery hypothesis based unsupervised pre-training for model compression in federated learning
Sohei Itahara, Takayuki Nishio, Masahiro Morikura, and Koji Yamamoto · 2004
Earlier work this paper cites.
Ang Li, Jingwei Sun, Binghui Wang, Lin Duan, Sicheng Li, Yiran Chen, and Hai Li · 2008
Earlier work this paper cites.
Song Han, Huizi Mao, and William J Dally · 2015
Earlier work this paper cites.
Network trimming: A data-driven neuron pruning approach towards efficient deep architectures
Hengyuan Hu, Rui Peng, Yu-Wing Tai, and Chi-Keung Tang · 2016
Earlier work this paper cites.
Learning structured sparsity in deep neural networks
Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li · 2016
Earlier work this paper cites.
Inductive representation learning on large graphs
William L. Hamilton, Zhitao Ying, and Jure Leskovec · 2017
Earlier work this paper cites.
Channel pruning for accelerating very deep neural networks
Yihui He, Xiangyu Zhang, and Jian Sun · 2017
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks
Thomas N. Kipf and Max Welling · 2017
Earlier work this paper cites.
Pruning filters for efficient convnets
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf · 2017
Earlier work this paper cites.
Learning efficient convolutional networks through network slimming
Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang · 2017
Earlier work this paper cites.
The lottery ticket hypothesis: Finding sparse, trainable neural networks
Jonathan Frankle and Michael Carbin · 2018
Earlier work this paper cites.
Efficient sparse-matrix multi-vector product on gpus
Changwan Hong, Aravind Sukumaran-Rajam, Bortik Bandyopadhyay, Jinsung Kim, Süreyya Emre Kurt, Israt Nisa, Shivani Sabhlok, Ümit V. Çatalyürek, Srinivasan Parthasarathy, and P. Sadayappan · 2018
Earlier work this paper cites.
A systematic DNN weight pruning framework using alternating direction method of multipliers
Tianyun Zhang, Shaokai Ye, Kaiqi Zhang, Jian Tang, Wujie Wen, Makan Fardad, and Yanzhi Wang · 2018
Earlier work this paper cites.
Efficient and effective sparse LSTM on FPGA with bank-balanced sparsity
Shijie Cao, Chen Zhang, Zhuliang Yao, Wencong Xiao, Lanshun Nie, De-chen Zhan, Yunxin Liu, Ming Wu, and Lintao Zhang · 2019
Earlier work this paper cites.
Evaluating lottery tickets under distributional shifts
Shrey Desai, Hongyuan Zhan, and Ahmed Aly · 2019
Earlier work this paper cites.
Rethinking the value of network pruning
Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell · 2019
Earlier work this paper cites.
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers
Ari S. Morcos, Haonan Yu, Michela Paganini, and Yuandong Tian · 2019
Earlier work this paper cites.
ADMM-NN: an algorithm-hardware co-design framework of dnns using alternating direction methods of multipliers
Ao Ren, Tianyun Zhang, Shaokai Ye, Jiayu Li, Wenyao Xu, Xuehai Qian, Xue Lin, and Yanzhi Wang · 2019
Earlier work this paper cites.
Using winning lottery tickets in transfer learning for convolutional neural networks
Ryan Van Soelen and John W Sheppard · 2019
Earlier work this paper cites.
Playing the lottery with rewards and multiple languages: lottery tickets in rl and nlp
Haonan Yu, Sergey Edunov, Yuandong Tian, and Ari S Morcos · 2019
Earlier work this paper cites.
Deconstructing lottery tickets: Zeros, signs, and the supermask
Hattie Zhou, Janice Lan, Rosanne Liu, and Jason Yosinski · 2019
Earlier work this paper cites.
Losing heads in the lottery: Pruning transformer attention in neural machine translation
Maximiliana Behnke and Kenneth Heafield · 2020
Earlier work this paper cites.
Successfully applying the stabilized lottery ticket hypothesis to the transformer architecture
Christopher Brix, Parnia Bahar, and Hermann Ney · 2020
Earlier work this paper cites.
Language models are few-shot learners
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei · 2020
Earlier work this paper cites.
Rigging the lottery: Making all tickets winners
Utku Evci, Trevor Gale, Jacob Menick, Pablo Samuel Castro, and Erich Elsen · 2020
Earlier work this paper cites.
Resprop: Reuse sparsified backpropagation
Negar Goli and Tor M. Aamodt · 2020
Earlier work this paper cites.
Denoising diffusion probabilistic models
Jonathan Ho, Ajay Jain, and Pieter Abbeel · 2020
Earlier work this paper cites.
Predicting economic growth by region embedding: A multigraph convolutional network approach
Bo Hui, Da Yan, Wei-Shinn Ku, and Wenlu Wang · 2020
Earlier work this paper cites.
Proving the lottery ticket hypothesis: Pruning is all you need
Eran Malach, Gilad Yehudai, Shai Shalev-Shwartz, and Ohad Shamir · 2020
Earlier work this paper cites.
Logarithmic pruning is all you need
Laurent Orseau, Marcus Hutter, and Omar Rivasplata · 2020
Earlier work this paper cites.
Optimal lottery tickets via subset sum: Logarithmic over-parameterization is sufficient
Ankit Pensia, Shashank Rajput, Alliot Nagle, Harit Vishwakarma, and Dimitris S. Papailiopoulos · 2020
Earlier work this paper cites.
When bert plays the lottery, all tickets are winning
Sai Prasanna, Anna Rogers, and Anna Rumshisky · 2020
Earlier work this paper cites.
Sparse weight activation training
Md Aamir Raihan and Tor M. Aamodt · 2020
Cited alongside, same era.
What’s hidden in a randomly weighted neural network?
Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari · 2020
Cited alongside, same era.
Comparing rewinding and fine-tuning in neural network pruning
Alex Renda, Jonathan Frankle, and Michael Carbin · 2020
Cited alongside, same era.
Winning the lottery with continuous sparsification
Pedro Savarese, Hugo Silva, and Michael Maire · 2020
Cited alongside, same era.
Sanity-checking pruning methods: Random tickets can win the jackpot
Jingtong Su, Yihang Chen, Tianle Cai, Tianhao Wu, Ruiqi Gao, Liwei Wang, and Jason D. Lee · 2020
Cited alongside, same era.
Learning sparse sharing architectures for multiple tasks
Tianxiang Sun, Yunfan Shao, Xiaonan Li, Pengfei Liu, Hang Yan, Xipeng Qiu, and Xuanjing Huang · 2020
CHEX: channel exploration for CNN model compression
Zejiang Hou, Minghai Qin, Fei Sun, Xiaolong Ma, Kun Yuan, Yi Xu, Yen-Kuang Chen, Rong Jin, Yuan Xie, and Sun-Yuan Kung · 2022
Later among the works it cites.
The lottery ticket hypothesis for self-attention in convolutional neural network
Zhongzhan Huang, Senwei Liang, Mingfu Liang, Wei He, Haizhao Yang, and Liang Lin · 2022
Later among the works it cites.
How well do sparse imagenet models transfer?
Eugenia Iofinova, Alexandra Peste, Mark Kurtz, and Dan Alistarh · 2022
Later among the works it cites.
Exploring lottery ticket hypothesis in spiking neural networks
Youngeun Kim, Yuhang Li, Hyoungseob Park, Yeshwanth Venkatesha, Ruokai Yin, and Priyadarshini Panda · 2022
Later among the works it cites.
Effective model sparsification by scheduled grow-and-prune methods
Xiaolong Ma, Minghai Qin, Fei Sun, Zejiang Hou, Kun Yuan, Yi Xu, Yanzhi Wang, Yen-Kuang Chen, Rong Jin, and Yuan Xie · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Pruning neural networks without any data by iteratively conserving synaptic flow
Hidenori Tanaka, Daniel Kunin, Daniel L. K. Yamins, and Surya Ganguli · 2020
Cited alongside, same era.
Picking winning tickets before training by preserving gradient flow
Chaoqi Wang, Guodong Zhang, and Roger B. Grosse · 2020
Cited alongside, same era.
Greedy optimization provably wins the lottery: Logarithmic number of winning tickets is enough
Mao Ye, Lemeng Wu, and Qiang Liu · 2020
Cited alongside, same era.
Drawing early-bird tickets: Toward more efficient training of deep networks
Haoran You, Chaojian Li, Pengfei Xu, Yonggan Fu, Yue Wang, Xiaohan Chen, Richard G. Baraniuk, Zhangyang Wang, and Yingyan Lin · 2020
Cited alongside, same era.
A generalized lottery ticket hypothesis
Ibrahim M. Alabdulmohsin, Larisa Markeeva, Daniel Keysers, and Ilya O. Tolstikhin · 2021
Cited alongside, same era.
Long live the lottery: The existence of winning tickets in lifelong learning
Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang · 2021
Cited alongside, same era.
Fedltn: Federated learning for sparse and personalized lottery ticket networks
Vaikkunth Mugunthan, Eric Lin, Vignesh Gokul, Christian Lau, Lalana Kagal, and Steven D. Pieper · 2022
Later among the works it cites.
SOSP: efficiently capturing global correlations by second-order structured pruning
Manuel Nonnenmacher, Thomas Pfeil, Ingo Steinwart, and David Reeb · 2022
Later among the works it cites.
A study on the ramanujan graph property of winning lottery tickets
Bithika Pal, Arindam Biswas, Sudeshna Kolay, Pabitra Mitra, and Biswajit Basu · 2022
Later among the works it cites.
Unmasking the lottery ticket hypothesis: What’s encoded in a winning ticket’s mask?
Mansheej Paul, Feng Chen, Brett W Larsen, Jonathan Frankle, Surya Ganguli, and Gintare Karolina Dziugaite · 2022
Later among the works it cites.
Winning the lottery ahead of time: Efficient early network pruning
John Rachwan, Daniel Zügner, Bertrand Charpentier, Simon Geisler, Morgane Ayle, and Stephan Günnemann · 2022
Later among the works it cites.
Analyzing lottery ticket hypothesis from pac-bayesian theory perspective
Keitaro Sakamoto and Issei Sato · 2022
Later among the works it cites.
Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations
Steffen Schotthöfer, Emanuele Zangrando, Jonas Kusch, Gianluca Ceruti, and Francesco Tudisco · 2022
Later among the works it cites.
Win the lottery ticket via fourier analysis: Frequencies guided network pruning
Yuzhang Shang, Bin Duan, Ziliang Zong, Liqiang Nie, and Yan Yan · 2022
Later among the works it cites.
When to prune? a policy towards early structural pruning
Maying Shen, Pavlo Molchanov, Hongxu Yin, and Jose M Alvarez · 2022
Later among the works it cites.
Does dataset lottery ticket hypothesis exist?
Zhiqiang Shen and Eric Xing · 2022
Later among the works it cites.
On lottery tickets and minimal task representations in deep reinforcement learning
Marc Aurel Vischer, Robert Tjarko Lange, and Henning Sprekeler · 2022
Later among the works it cites.
Pruning adatperfusion with lottery ticket hypothesis
Jiarun Wu, Qingliang Chen, Zeguan Xiao, Yuliang Gu, and Mengsi Sun · 2022
Later among the works it cites.
Efficient adversarial training with robust early-bird tickets
Zhiheng Xi, Rui Zheng, Tao Gui, Qi Zhang, and Xuanjing Huang · 2022
Later among the works it cites.
Robust lottery tickets for pre-trained language models
Rui Zheng, Rong Bao, Yuhao Zhou, Di Liang, Sirui Wang, Wei Wu, Tao Gui, Qi Zhang, and Xuanjing Huang · 2022
Later among the works it cites.
Lottery aware sparsity hunting: Enabling federated learning on resource-limited edge, 2023
Sara Babakniya, Souvik Kundu, Saurav Prakash, Yue Niu, and Salman Avestimehr · 2023
Later among the works it cites.
Rethinking graph lottery tickets: Graph sparsity matters
Bo Hui, Da Yan, Xiaolong Ma, and Wei-Shinn Ku · 2023
Later among the works it cites.
Instant soup: Cheap pruning ensembles in A single pass can draw lottery tickets from large models
Ajay Kumar Jaiswal, Shiwei Liu, Tianlong Chen, Ying Ding, and Zhangyang Wang · 2023
Later among the works it cites.
Successfully applying lottery ticket hypothesis to diffusion model
Chao Jiang, Bo Hui, Bohan Liu, and Da Yan · 2023
Later among the works it cites.
Accelerable lottery tickets with the mixed-precision quantization
Zhangheng Li, Yu Gong, Zhenyu Zhang, Xingyun Xue, Tianlong Chen, Yi Liang, Bo Yuan, and Zhangyang Wang · 2023
Later among the works it cites.
Can unstructured pruning reduce the depth in deep neural networks?
Zhu Liao, Victor Quétu, Van-Tam Nguyen, and Enzo Tartaglione · 2023
Later among the works it cites.
Memory-friendly scalable super-resolution via rewinding lottery ticket hypothesis
Jin Lin, Xiaotong Luo, Ming Hong, Yanyun Qu, Yuan Xie, and Zongze Wu · 2023
Later among the works it cites.
Gradient-free structured pruning with unlabeled data
Azade Nova, Hanjun Dai, and Dale Schuurmans · 2023
Later among the works it cites.
Data level lottery ticket hypothesis for vision transformers
Xuan Shen, Zhenglun Kong, Minghai Qin, Peiyan Dong, Geng Yuan, Xin Meng, Hao Tang, Xiaolong Ma, and Yanzhi Wang · 2023
Later among the works it cites.
Efficient federated learning with enhanced privacy via lottery ticket pruning in edge computing
Yifan Shi, Kang Wei, Li Shen, Jun Li, Xueqian Wang, Bo Yuan, and Song Guo · 2023
Later among the works it cites.
Fair scratch tickets: Finding fair sparse networks without weight training
Pengwei Tang, Wei Yao, Zhicong Li, and Yong Liu · 2023
Later among the works it cites.
Can we find strong lottery tickets in generative models?
Sangyeop Yeo, Yoojin Jang, Jy-yong Sohn, Dongyoon Han, and Jaejun Yoo · 2023
Later among the works it cites.
Lottery pools: Winning more by interpolating tickets without increasing training or inference cost
Lu Yin, Shiwei Liu, Meng Fang, Tianjin Huang, Vlado Menkovski, and Mykola Pechenizkiy · 2023
Later among the works it cites.
GOHSP: A unified framework of graph and optimization-based heterogeneous structured pruning for vision transformer
Miao Yin, Burak Uzkent, Yilin Shen, Hongxia Jin, and Bo Yuan · 2023
Later among the works it cites.
Masks, signs, and learning rate rewinding
Advait Harshal Gadhikar and Rebekka Burkholz · 2024
Closest in time.
Graph lottery ticket automated
Guibin Zhang, Kun Wang, Wei Huang, Yanwei Yue, Yang Wang, Roger Zimmermann, Aojun Zhou, Dawei Cheng, Jin Zeng, and Yuxuan Liang · 2024
Closest in time.