Fetching the paper…
Reading the bibliography…
Existing approaches to federated learning suffer from a communication bottleneck as well as convergence issues due to sparse client participation.
Error feedback fixes signsgd and other gradient compression schemes
Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian U Stich, and Martin Jaggi · 1901
Earlier work this paper cites.
Finding repeated elements
Jayadev Misra and David Gries · 1982
Earlier work this paper cites.
The space complexity of approximating the frequency moments
Noga Alon, Yossi Matias, and Mario Szegedy · 1999
Earlier work this paper cites.
Finding frequent items in data streams
Moses Charikar, Kevin Chen, and Martin Farach-Colton · 2002
Earlier work this paper cites.
Maintaining stream statistics over sliding windows
Mayur Datar, Aristides Gionis, Piotr Indyk, and Rajeev Motwani · 2002
Earlier work this paper cites.
Data streams: Algorithms and applications
Shanmugavelayutham Muthukrishnan et al · 2005
Earlier work this paper cites.
Smooth histograms for sliding windows
Vladimir Braverman and Rafail Ostrovsky · 2007
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al · 2009
Earlier work this paper cites.
The use of mobile phones as a data collection tool: a report from a household survey in south africa
Mark Tomlinson, Wesley Solomon, Yages Singh, Tanya Doherty, Mickey Chopra, Petrida Ijumba, Alexander C Tsai, and Debra Jackson · 2009
Earlier work this paper cites.
Mobile data offloading: How much can wifi deliver?
Kyunghan Lee, Joohyun Lee, Yung Yi, Injong Rhee, and Song Chong · 2010
Earlier work this paper cites.
Mobile intelligence , volume 69
Laurence T Yang, BW Augustinus, Jianhua Ma, Ling Tan, and Bala Srinivasan · 2010
Earlier work this paper cites.
Large scale distributed deep networks
Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Marc’aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, et al · 2012
Earlier work this paper cites.
Speed measurements of residential internet access
Oana Goga and Renata Teixeira · 2012
Earlier work this paper cites.
Origins of power-law degree distribution in the heterogeneity of human activity in social networks
Lev Muchnik, Sen Pei, Lucas C Parra, Saulo DS Reis, José S Andrade Jr, Shlomo Havlin, and Hernán A Makse · 2013
Earlier work this paper cites.
On the importance of initialization and momentum in deep learning
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton · 2013
Earlier work this paper cites.
How to catch l2-heavy-hitters on sliding windows
Vladimir Braverman, Ran Gelles, and Rafail Ostrovsky · 2014
Earlier work this paper cites.
Zero-one laws for sliding windows and universal sketches
Vladimir Braverman, Rafail Ostrovsky, and Alan Roytman · 2015
Earlier work this paper cites.
Streaming algorithms for halo finders
Zaoxing Liu, Nikita Ivkin, Lin Yang, Mark Neyrinck, Gerard Lemson, Alexander Szalay, Vladimir Braverman, Tamas Budavari, Randal Burns, and Xin Wang · 2015
Earlier work this paper cites.
Practical secure aggregation for federated learning on user-held data
Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth · 2016
Earlier work this paper cites.
Federated learning: Strategies for improving communication efficiency, 2016
Jakub Konecny, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon · 2016
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data
H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al · 2016
Cited alongside, same era.
Edge computing: Vision and challenges
Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu · 2016
Cited alongside, same era.
Qsgd: Communication-efficient sgd via gradient quantization and encoding
Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic · 2017
Cited alongside, same era.
Bptree: an ℓ 2 \ell_{2} heavy hitters algorithm using constant memory
Vladimir Braverman, Stephen R Chestnut, Nikita Ivkin, Jelani Nelson, Zhengyu Wang, and David P Woodruff · 2017
Cited alongside, same era.
Emnist: Extending mnist to handwritten letters
G. Cohen, S. Afshar, J. Tapson, and A. van Schaik · 2017
Cited alongside, same era.
On the convergence of federated optimization in heterogeneous networks
Anit Kumar Sahu, Tian Li, Maziar Sanjabi, Manzil Zaheer, Ameet Talwalkar, and Virginia Smith · 2018
Later among the works it cites.
Sparsified sgd with memory
Sebastian U Stich, Jean-Baptiste Cordonnier, and Martin Jaggi · 2018
Later among the works it cites.
Cooperative sgd: A unified framework for the design and analysis of communication-efficient sgd algorithms, 2018
Jianyu Wang and Gauri Joshi · 2018
Later among the works it cites.
Gradient sparsification for communication-efficient distributed optimization
Jianqiao Wangni, Jialei Wang, Ji Liu, and Tong Zhang · 2018
Later among the works it cites.
Applied federated learning: Improving google keyboard query suggestions
Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ramage, and Francoise Beaufays · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Robin C. Geyer, Tassilo Klein, and Moin Nabi · 2017
Cited alongside, same era.
Accurate, large minibatch sgd: Training imagenet in 1 hour
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He · 2017
Cited alongside, same era.
Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Richard Nock, Giorgio Patrini, Guillaume Smith, and Brian Thorne · 2017
Cited alongside, same era.
Deep gradient compression: Reducing the communication bandwidth for distributed training
Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J Dally · 2017
Cited alongside, same era.
How to backdoor federated learning, 2018
Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov · 2018
Cited alongside, same era.
signsgd: Compressed optimisation for non-convex problems
Jeremy Bernstein, Yu-Xiang Wang, Kamyar Azizzadenesheli, and Anima Anandkumar · 2018
Cited alongside, same era.
Analyzing federated learning through an adversarial lens
Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo · 2018
Cited alongside, same era.
Personalizing dialogue agents: I have a dog, do you have pets too?, 2018
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston · 2018
Later among the works it cites.
Federated learning with non-iid data, 2018
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra · 2018
Later among the works it cites.
Advances and open problems in federated learning, 2019
Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D’Oliveira, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konecny, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao · 2019
Later among the works it cites.
Federated learning for keyword spotting
David Leroy, Alice Coucke, Thibaut Lavril, Thibault Gisselbrecht, and Joseph Dureau · 2019
Later among the works it cites.
Privacy for free: Communication-efficient learning with differential privacy using sketches
Tian Li, Zaoxing Liu, Vyas Sekar, and Virginia Smith · 2019
Later among the works it cites.
How to train your resnet, Nov 2019
David Page · 2019
Later among the works it cites.
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala · 2019
Later among the works it cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever · 2019
Later among the works it cites.
Understanding top-k sparsification in distributed deep learning
Shaohuai Shi, Xiaowen Chu, Ka Chun Cheung, and Simon See · 2019
Later among the works it cites.
Compressing gradient optimizers via count-sketches
Ryan Spring, Anastasios Kyrillidis, Vijai Mohan, and Anshumali Shrivastava · 2019
Later among the works it cites.
Adaptive federated learning in resource constrained edge computing systems
Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K Leung, Christian Makaya, Ting He, and Kevin Chan · 2019
Later among the works it cites.
How to build a state-of-the-art conversational ai with transfer learning, May 2019
Thomas Wolf · 2019
Later among the works it cites.
Huggingface’s transformers: State-of-the-art natural language processing
Thomas Wolf, L Debut, V Sanh, J Chaumond, C Delangue, A Moi, P Cistac, T Rault, R Louf, M Funtowicz, et al · 2019
Later among the works it cites.
Communication-efficient distributed blockwise momentum sgd with error-feedback
Shuai Zheng, Ziyue Huang, and James Kwok · 2019
Later among the works it cites.