Fetching the paper…
Reading the bibliography…
When the available hardware cannot meet the memory and compute requirements to efficiently train high performing machine learning models, a compromise in either the training quality or the model complexity is needed.
Direct solutions of sparse network equations by optimally ordered triangular factorization
W.F. Tinney and J.W. Walker · 1967
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al · 2009
Earlier work this paper cites.
Distilling the knowledge in a neural network, 2015
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Dynamic network surgery for efficient dnns
Yiwen Guo, Anbang Yao, and Yurong Chen · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Federated learning: Strategies for improving communication efficiency
Jakub Konečnỳ, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon · 2016
Earlier work this paper cites.
Cambricon-X: An Accelerator for Sparse Neural Networks
S. Zhang, Z. Du, L. Zhang, H. Lan, S. Liu, L. Li, Q. Guo, T. Chen, and Y. Chen · 2016
Earlier work this paper cites.
Emnist: Extending mnist to handwritten letters
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik · 2017
Earlier work this paper cites.
Large scale population assessment of physical activity using wrist worn accelerometers: The uk biobank study
Aiden Doherty, Dan Jackson, Nils Hammerla, Thomas Plötz, Patrick Olivier, Malcolm H Granat, Tom White, Vincent T Van Hees, Michael I Trenell, Christoper G Owen, et al · 2017
Earlier work this paper cites.
Channel pruning for accelerating very deep neural networks
Yihui He, Xiangyu Zhang, and Jian Sun · 2017
Earlier work this paper cites.
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
Jianxin Wu Jian-Hao Luo and Weiyao Lin · 2017
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas · 2017
Earlier work this paper cites.
Variational dropout sparsifies deep neural networks
Dmitry Molchanov, Arsenii Ashukha, and Dmitry Vetrov · 2017
Earlier work this paper cites.
Block-sparse gpu kernels, 2017
A. Radford S. Gray and D. P. Kingma · 2017
Earlier work this paper cites.
meProp: Sparsified back propagation for accelerated deep learning with reduced overfitting
Xu Sun, Xuancheng Ren, Shuming Ma, and Houfeng Wang · 2017
Earlier work this paper cites.
Beyond Filters: Compact Feature Map for Portable Deep Model
Yunhe Wang, Chang Xu, Chao Xu, and Dacheng Tao · 2017
Earlier work this paper cites.
Leaf: A benchmark for federated settings
Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečnỳ, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar · 2018
Earlier work this paper cites.
Federated learning for mobile keyboard prediction, 2018
Andrew Hard, Chloé M Kiddon, Daniel Ramage, Francoise Beaufays, Hubert Eichner, Kanishka Rao, Rajiv Mathews, and Sean Augenstein · 2018
Earlier work this paper cites.
Applied machine learning at facebook: A datacenter infrastructure perspective
Kim Hazelwood, Sarah Bird, David Brooks, Soumith Chintala, Utku Diril, Dmytro Dzhulgakov, Mohamed Fawzy, Bill Jia, Yangqing Jia, Aditya Kalro, James Law, Kevin Lee, Jason Lu, Pieter Noordhuis, Misha Smelyanskiy, Liang Xiong, and Xiaodong Wang · 2018
Earlier work this paper cites.
Soft filter pruning for accelerating deep convolutional neural networks
Yang He, Guoliang Kang, Xuanyi Dong, Yanwei Fu, and Yi Yang · 2018
Earlier work this paper cites.
Efficient sparse-matrix multi-vector product on gpus
Changwan Hong, Aravind Sukumaran-Rajam, Bortik Bandyopadhyay, Jinsung Kim, Süreyya Emre Kurt, Israt Nisa, Shivani Sabhlok, Ümit V. Çatalyürek, Srinivasan Parthasarathy, and P. Sadayappan · 2018
Cited alongside, same era.
Federated optimization in heterogeneous networks
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith · 2018
Cited alongside, same era.
Sbnet: Sparse blocks network for fast inference, 2018
Mengye Ren, Andrei Pokrovsky, Bin Yang, and Raquel Urtasun · 2018
Cited alongside, same era.
Atomo: Communication-efficient learning via atomic sparsification
Hongyi Wang, Scott Sievert, Zachary Charles, Shengchao Liu, Stephen Wright, and Dimitris Papailiopoulos · 2018
Cited alongside, same era.
Speech commands: A dataset for limited-vocabulary speech recognition
Training keyword spotting models on non-iid data with federated learning, 2020
Andrew Hard, Kurt Partridge, Cameron Nguyen, Niranjan Subrahmanya, Aishanee Shah, Pai Zhu, Ignacio Lopez Moreno, and Rajiv Mathews · 2020
Later among the works it cites.
Group knowledge transfer: Federated learning of large cnns at the edge
Chaoyang He, Murali Annavaram, and Salman Avestimehr · 2020
Later among the works it cites.
Ensemble distillation for robust model fusion in federated learning
Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi · 2020
Later among the works it cites.
Sparse weight activation training
Md Aamir Raihan and Tor M Aamodt · 2020
Later among the works it cites.
Tensaurus: A Versatile Accelerator for Mixed Sparse-Dense Tensor Computations
N. Srivastava, H. Jin, S. Smith, H. Rong, D. Albonesi, and Z. Zhang · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Pete Warden · 2018
Cited alongside, same era.
Nisp: Pruning networks using neuron importance score propagation
R. Yu, A. Li, C. Chen, J. Lai, V. I. Morariu, X. Han, M. Gao, C. Lin, and L. S. Davis · 2018
Cited alongside, same era.
Hello edge: Keyword spotting on microcontrollers, 2018
Yundong Zhang, Naveen Suda, Liangzhen Lai, and Vikas Chandra · 2018
Cited alongside, same era.
Federated learning with personalization layers, 2019
Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary · 2019
Cited alongside, same era.
Expanding the reach of federated learning by reducing client resource requirements, 2019
Sebastian Caldas, Jakub Konečny, H. Brendan McMahan, and Ameet Talwalkar · 2019
Cited alongside, same era.
Adaptive sparse tiling for sparse matrix multiplication
Changwan Hong, Aravind Sukumaran-Rajam, Israt Nisa, Kunal Singh, and P. Sadayappan · 2019
Cited alongside, same era.
Measuring the effects of non-identical data distribution for federated visual classification
Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown · 2019
Cited alongside, same era.
Model pruning enables efficient federated learning on edge devices
Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K Leung, and Leandros Tassiulas · 2019
Cited alongside, same era.
Segblocks: Block-based dynamic resolution networks for real-time segmentation, 2020
Thomas Verelst and Tinne Tuytelaars · 2020
Later among the works it cites.
Sparsert: Accelerating unstructured sparsity on gpus for deep learning inference
Ziheng Wang · 2020
Later among the works it cites.
Block-sparse cnn: towards a fast and memory-efficient framework for convolutional neural networks
N. Wen, R. Guo, B. He, Yong Fan, and Ding Ma · 2020
Later among the works it cites.
Accelerating sparse matrix–matrix multiplication with gpu tensor cores
Orestis Zachariadis, Nitin Satpute, Juan Gómez-Luna, and Joaquín Olivares · 2020
Later among the works it cites.
Fine-tuning is fine in federated learning, 2021
Gary Cheng, Karan Chadha, and John Duchi · 2021
Later among the works it cites.
Scaling federated learning for fine-tuning of large language models, 2021
Agrin Hilmkil, Sebastian Callh, Matteo Barbieri, Leon René Sütfeld, Edvin Listo Zec, and Olof Mogren · 2021
Later among the works it cites.
Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks, 2021
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste · 2021
Later among the works it cites.
Fjord: Fair and accurate federated learning under heterogeneous targets with ordered dropout
Samuel Horvath, Stefanos Laskaridis, Mario Almeida, Ilias Leontiadis, Stylianos I Venieris, and Nicholas D Lane · 2021
Later among the works it cites.
Hierarchical quantized federated learning: Convergence analysis and system design, 2021
Lumin Liu, Jun Zhang, Shenghui Song, and Khaled B. Letaief · 2021
Later among the works it cites.
A first look into the carbon footprint of federated learning
Xinchi Qiu, Titouan Parcollet, Javier Fernandez-Marques, Pedro Porto Buarque de Gusmao, Daniel J Beutel, Taner Topal, Akhil Mathur, and Nicholas D Lane · 2021
Later among the works it cites.
Adaptive federated optimization
Sashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and Hugh Brendan McMahan · 2021
Later among the works it cites.
Communication efficiency in federated learning: Achievements and challenges, 2021
Osama Shahid, Seyedamin Pouriyeh, Reza M. Parizi, Quan Z. Sheng, Gautam Srivastava, and Liang Zhao · 2021
Later among the works it cites.
Sparsednn: Fast sparse deep learning inference on cpus, 2021
Ziheng Wang · 2021
Later among the works it cites.
Data-free knowledge distillation for heterogeneous federated learning, 2021
Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou · 2021
Later among the works it cites.