Fetching the paper…

Reading the bibliography…

2020

Evaluating the Communication Efficiency in Federated Learning Algorithms

Asad, Muhammad, Moustafa, Ahmed, Ito, Takayuki et al.

Original
PDF
Post on X

Understand

In the era of advanced technologies, mobile devices are equipped with computing and sensing capabilities that gather excessive amounts of data.

These amounts of data are suitable for training different learning models.
Cooperated with advancements in Deep Learning (DL), these learning models empower numerous useful applications, e.g., image processing, speech recognition, healthcare, vehicular network and many more.
Traditionally, Machine Learning (ML) approaches require data to be centralised in cloud-based data-centres.

Evaluating the Communication Efficiency in Federated Learning Algorithms · Around

Built on

N. Strom, “Scalable distributed dnn training using commodity gpu cloud computing,” in Sixteenth Annual Conference of the International Speech Communication Association , 2015
2015
Earlier work this paper cites.
H. B. McMahan, E. Moore, D. Ramage, S. Hampson et al. , “Communication-efficient learning of deep networks from decentralized data,” arXiv preprint arXiv:1602.05629 , 2016
Original
2016
Earlier work this paper cites.
H. B. McMahan, E. Moore, D. Ramage, and B. A. y Arcas, “Federated learning of deep networks using model averaging,” 2016
2016
Earlier work this paper cites.
Y. Lin, S. Han, H. Mao, Y. Wang, and W. J. Dally, “Deep gradient compression: Reducing the communication bandwidth for distributed training,” arXiv preprint arXiv:1712.01887 , 2017
Original
2017
Earlier work this paper cites.
A. F. Aji and K. Heafield, “Sparse communication for distributed gradient descent,” arXiv preprint arXiv:1704.05021 , 2017
Original
2017
Earlier work this paper cites.
W. Wen, C. Xu, F. Yan, C. Wu, Y. Wang, Y. Chen, and H. Li, “Terngrad: Ternary gradients to reduce communication in distributed deep learning,” in Advances in neural information processing systems , 2017, pp. 1509–1519
2017
Earlier work this paper cites.
D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. Vojnovic, “Qsgd: Communication-efficient sgd via gradient quantization and encoding,” in Advances in Neural Information Processing Systems , 2017, pp. 1709–1720
2017
Earlier work this paper cites.
Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated learning with non-iid data,” arXiv preprint arXiv:1806.00582 , 2018
Original
2018
Earlier work this paper cites.

Similar

T. Ryffel, A. Trask, M. Dahl, B. Wagner, J. Mancuso, D. Rueckert, and J. Passerat-Palmbach, “A generic framework for privacy preserving deep learning,” arXiv preprint arXiv:1811.04017 , 2018
Original
2018
Cited alongside, same era.
S. Caldas, P. Wu, T. Li, J. Konečnỳ, H. B. McMahan, V. Smith, and A. Talwalkar, “Leaf: A benchmark for federated settings,” arXiv preprint arXiv:1812.01097 , 2018
Original
2018
Cited alongside, same era.
J. Bernstein, Y.-X. Wang, K. Azizzadenesheli, and A. Anandkumar, “signsgd: Compressed optimisation for non-convex problems,” arXiv preprint arXiv:1802.04434 , 2018
Original
2018
Cited alongside, same era.
J. Bernstein, J. Zhao, K. Azizzadenesheli, and A. Anandkumar, “signsgd with majority vote is communication efficient and fault tolerant,” arXiv , 2018
2018
Cited alongside, same era.

Then

A. Reisizadeh, A. Mokhtari, H. Hassani, A. Jadbabaie, and R. Pedarsani, “Fedpaq: A communication-efficient federated learning method with periodic averaging and quantization,” arXiv preprint arXiv:1909.13014 , 2019
Original
2019
Later among the works it cites.
F. Ang, L. Chen, N. Zhao, Y. Chen, W. Wang, and F. R. Yu, “Robust federated learning with noisy communication,” arXiv preprint arXiv:1911.00251 , 2019
Original
2019
Later among the works it cites.
M. Nasr, R. Shokri, and A. Houmansadr, “Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning,” in 2019 IEEE Symposium on Security and Privacy (SP) . IEEE, 2019, pp. 739–753
2019
Later among the works it cites.
T. Nishio and R. Yonetani, “Client selection for federated learning with heterogeneous resources in mobile edge,” in ICC 2019-2019 IEEE International Conference on Communications (ICC) . IEEE, 2019, pp. 1–7
2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

H. Wang, S. Sievert, S. Liu, Z. Charles, D. Papailiopoulos, and S. Wright, “Atomo: Communication-efficient learning via atomic sparsification,” in Advances in Neural Information Processing Systems , 2018, pp. 9850–9861

2018

Cited alongside, same era.

S. Caldas, J. Konečny, H. B. McMahan, and A. Talwalkar, “Expanding the reach of federated learning by reducing client resource requirements,” arXiv preprint arXiv:1812.07210 , 2018

2018

Cited alongside, same era.

S. Wiedemann, K.-R. Müller, and W. Samek, “Compact and computationally efficient representation of deep neural networks,” IEEE transactions on neural networks and learning systems , 2019

2019

Cited alongside, same era.

W. Y. B. Lim, N. C. Luong, D. T. Hoang, Y. Jiao, Y.-C. Liang, Q. Yang, D. Niyato, and C. Miao, “Federated learning in mobile edge networks: A comprehensive survey,” arXiv preprint arXiv:1909.11875 , 2019

Original

2019

Cited alongside, same era.

“Google, “tensorflow federated: Machine learning on decentralized data.”

Cited in the paper.

L. Wang, W. Wang, and B. Li, “Cmfl: Mitigating communication overhead for federated learning.”

Cited in the paper.

K. Hsieh, A. Harlap, N. Vijaykumar, D. Konomis, G. R. Ganger, P. B. Gibbons, and O. Mutlu, “Gaia: Geo-distributed machine learning approaching { \{ LAN } \} speeds,” in 14th { \{ USENIX } \} Symposium on Networked Systems Design and Implementation ( { \{ NSDI } \} 17) , 2017, pp. 629–647

Cited in the paper.

F. Sattler, S. Wiedemann, K.-R. Müller, and W. Samek, “Robust and communication-efficient federated learning from non-iid data,” arXiv preprint arXiv:1903.02891 , 2019

2019

Later among the works it cites.

X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,” arXiv preprint arXiv:1907.02189 , 2019

2019

Later among the works it cites.

X. Yao, T. Huang, C. Wu, R.-x. Zhang, and L. Sun, “Federated learning with additional mechanisms on clients to reduce communication costs,” arXiv preprint arXiv:1908.05891 , 2019

2019

Later among the works it cites.