Model Pruning Enables Efficient Federated Learning on Edge Devices

Y. LeCun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” in Advances in neural information processing systems , 1990, pp. 598–605

1990

Earlier work this paper cites.

Y. LeCun, L. Bottou, Y. Bengio, P. Haffner et al. , “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998

1998

Earlier work this paper cites.

A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” Citeseer, Tech. Rep., 2009

2009

Earlier work this paper cites.

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition . Ieee, 2009, pp. 248–255

2009

Earlier work this paper cites.

M. Jaderberg, A. Vedaldi, and A. Zisserman, “Speeding up convolutional neural networks with low rank expansions,” in Proceedings of the British Machine Vision Conference. BMVA Press , 2014

2014

Earlier work this paper cites.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014

Original

2014

Earlier work this paper cites.

S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” in Advances in neural information processing systems , 2015, pp. 1135–1143

2015

Earlier work this paper cites.

S. Gupta, A. Agrawal, K. Gopalakrishnan, and P. Narayanan, “Deep learning with limited numerical precision,” in International Conference on Machine Learning , 2015, pp. 1737–1746

2015

Earlier work this paper cites.

G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015

Original

2015

Earlier work this paper cites.

I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning . MIT Press, 2016, http://www.deeplearningbook.org

2016

Earlier work this paper cites.

J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” arXiv preprint arXiv:1610.05492 , 2016

Original

2016

Earlier work this paper cites.

P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz, “Pruning convolutional neural networks for resource efficient inference,” arXiv preprint arXiv:1611.06440 , 2016

Original

2016

Earlier work this paper cites.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

2016

Earlier work this paper cites.

H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in AISTATS , 2017

2017

Earlier work this paper cites.

S. Narang, E. Elsen, G. Diamos, and S. Sengupta, “Exploring sparsity in recurrent neural networks,” in International Conference on Learning Representations , 2017

2017

Earlier work this paper cites.

M. Zhu and S. Gupta, “To prune, or not to prune: exploring the efficacy of pruning for model compression,” arXiv preprint arXiv:1710.01878 , 2017

Original

2017

Earlier work this paper cites.

S. Anwar, K. Hwang, and W. Sung, “Structured pruning of deep convolutional neural networks,” ACM Journal on Emerging Technologies in Computing Systems (JETC) , vol. 13, no. 3, pp. 1–18, 2017

2017

Earlier work this paper cites.

J. Lin, Y. Rao, J. Lu, and J. Zhou, “Runtime neural pruning,” in Proceedings of the 31st International Conference on Neural Information Processing Systems , 2017, pp. 2178–2188

2017

Earlier work this paper cites.

K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” in proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security , 2017, pp. 1175–1191

2017

Cited alongside, same era.

S. Caldas, J. Konečny, H. B. McMahan, and A. Talwalkar, “Expanding the reach of federated learning by reducing client resource requirements,” arXiv preprint arXiv:1812.07210 , 2018

Original

2018

Cited alongside, same era.

D. Alistarh, T. Hoefler, M. Johansson, N. Konstantinov, S. Khirirat, and C. Renggli, “The convergence of sparsified gradient methods,” in Advances in Neural Information Processing Systems , 2018, pp. 5973–5983

2018

Cited alongside, same era.

P. Jiang and G. Agrawal, “A linear speedup analysis of distributed deep learning with sparse and quantized communication,” in Advances in Neural Information Processing Systems 31 , S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., 2018, pp. 2525–2536

J. Wang and G. Joshi, “Adaptive communication strategies to achieve the best error-runtime trade-off in local-update sgd,” in Machine Learning and Systems (MLSys) , 2019

2019

Closest in time.

S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He, and K. Chan, “Adaptive federated learning in resource constrained edge computing systems,” IEEE Journal on Selected Areas in Communications , vol. 37, no. 6, pp. 1205–1221, June 2019

2019

Closest in time.

S. P. Karimireddy, S. Kale, M. Mohri, S. J. Reddi, S. U. Stich, and A. T. Suresh, “Scaffold: Stochastic controlled averaging for federated learning,” arXiv preprint arXiv:1910.06378 , 2019

Original

2019

Closest in time.

S. P. Karimireddy, Q. Rebjock, S. Stich, and M. Jaggi, “Error feedback fixes SignSGD and other gradient compression schemes,” in International Conference on Machine Learning , vol. 97, Jun. 2019, pp. 3252–3261

2019

Closest in time.