Built on

M. Mohri, G. Sivek, and A. T. Suresh, “Agnostic Federated Learning,”
Original
1902
Earlier work this paper cites.
N. Guha, A. Talwalkar, and V. Smith, “One-Shot Federated Learning,”
Original
1902
Earlier work this paper cites.
W. Zhu, P. Kairouz, B. McMahan, H. Sun, and W. Li, “Federated Heavy Hitters Discovery with Differential Privacy,”
Original
1902
Earlier work this paper cites.
H. Yu, R. Jin, and S. Yang, “On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization,”
Original
1905
Earlier work this paper cites.
M. Khodak, M.-F. Balcan, and A. Talwalkar, “Adaptive Gradient-Based Meta-Learning Methods,”
Original
1906
Earlier work this paper cites.
X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the Convergence of FedAvg on Non-IID Data,”
Original
1907
Earlier work this paper cites.
S. U. Stich, “Unified Optimal Analysis of the (Stochastic) Gradient Method,” Jul. 2019. [Online]. Available:
Original
1907
Earlier work this paper cites.
A. Fallah, A. Mokhtari, and A. Ozdaglar, “On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms,”
Original
1908
Earlier work this paper cites.
T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated Learning: Challenges, Methods, and Future Directions,”
Original
1908
Earlier work this paper cites.
A. Reisizadeh, A. Mokhtari, H. Hassani, A. Jadbabaie, and R. Pedarsani, “FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization,”
Original
1909
Earlier work this paper cites.
A. Khaled, K. Mishchenko, and P. Richtárik, “Tighter Theory for Local SGD on Identical and Heterogeneous Data,”
Original
1909
Earlier work this paper cites.
Y. Jiang, J. Konečný, K. Rush, and S. Kannan, “Improving Federated Learning Personalization via Model Agnostic Meta Learning,”
Original
1909
Earlier work this paper cites.
A. Khaled, K. Mishchenko, and P. Richtárik, “First Analysis of Local GD on Heterogeneous Data,”
Original
1909
Earlier work this paper cites.
S. P. Karimireddy
Original
1910
Earlier work this paper cites.
D. Li and J. Wang, “FedMD: Heterogenous Federated Learning via Model Distillation,”
Original
1910
Earlier work this paper cites.
F. Haddadpour and M. Mahdavi, “On the Convergence of Local Descent Methods in Federated Learning,”
Original
1910
Earlier work this paper cites.
X. Li, W. Yang, S. Wang, and Z. Zhang, “Communication-Efficient Local Decentralized SGD Methods,”
Original
1910
Earlier work this paper cites.

Similar

K. Pillutla, S. M. Kakade, and Z. Harchaoui, “Robust Aggregation for Federated Learning,”
Original
1912
Cited alongside, same era.
M. G. Arivazhagan, V. Aggarwal, A. K. Singh, and S. Choudhary, “Federated Learning with Personalization Layers,”
Original
1912
Cited alongside, same era.
J.-J. Moreau, “Propriétés des applications ‘prox’,”
1963
Cited alongside, same era.
C. Lemaréchal and C. Sagastizábal, “Practical Aspects of the Moreau–Yosida Regularization: Theoretical Preliminaries,”
1997
Cited alongside, same era.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-Based Learning Applied to Document Recognition,”
1998
Cited alongside, same era.

Then

V. Smith, C.-K. Chiang, M. Sanjabi, and A. Talwalkar, “Federated Multi-Task Learning,”
Original
2018
Later among the works it cites.
H. B. McMahan, D. Ramage, K. Talwar, and L. Zhang, “Learning Differentially Private Recurrent Language Models,”
Original
2018
Later among the works it cites.
A. Nichol, J. Achiam, and J. Schulman, “On First-Order Meta-Learning Algorithms,”
Original
2018
Later among the works it cites.
F. Chen, Z. Dong, Z. Li, and X. He, “Federated Meta-Learning for Recommendation,” Feb. 2018
2018
Later among the works it cites.
H. Lin, J. Mairal, and Z. Harchaoui, “Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice,”
Original
2018

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2020

Personalized Federated Learning with Moreau Envelopes

Dinh, Canh T., Tran, Nguyen H., Nguyen, Tuan Dung

Understand

Federated learning (FL) is a decentralized and privacy-preserving machine learning technique in which a group of clients collaborate with a server to learn a global model without sharing clients' data.

One challenge associated with FL is statistical diversity among clients, which restricts the global model from delivering good performance on each client's task.
To address this, we propose an algorithm for personalized FL (pFedMe) using Moreau envelopes as clients' regularized loss functions, which help decouple personalized model optimization from the global model learning in a bi-level problem stylized for personalized FL.
Theoretically, we show that pFedMe's convergence rate is state-of-the-art: achieving quadratic speedup for strongly convex and sublinear speedup of order 2/3 for smooth nonconvex objectives.

Built on

M. Mohri, G. Sivek, and A. T. Suresh, “Agnostic Federated Learning,”
Original
1902
Earlier work this paper cites.
N. Guha, A. Talwalkar, and V. Smith, “One-Shot Federated Learning,”
Original
1902
Earlier work this paper cites.
W. Zhu, P. Kairouz, B. McMahan, H. Sun, and W. Li, “Federated Heavy Hitters Discovery with Differential Privacy,”
Original
1902
Earlier work this paper cites.
H. Yu, R. Jin, and S. Yang, “On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization,”
Original
1905
Earlier work this paper cites.
M. Khodak, M.-F. Balcan, and A. Talwalkar, “Adaptive Gradient-Based Meta-Learning Methods,”
Original
1906
Earlier work this paper cites.
X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the Convergence of FedAvg on Non-IID Data,”
Original
1907
Earlier work this paper cites.
S. U. Stich, “Unified Optimal Analysis of the (Stochastic) Gradient Method,” Jul. 2019. [Online]. Available:
Original
1907
Earlier work this paper cites.
A. Fallah, A. Mokhtari, and A. Ozdaglar, “On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms,”
Original
1908
Earlier work this paper cites.
T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated Learning: Challenges, Methods, and Future Directions,”
Original
1908
Earlier work this paper cites.
A. Reisizadeh, A. Mokhtari, H. Hassani, A. Jadbabaie, and R. Pedarsani, “FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization,”
Original
1909
Earlier work this paper cites.
A. Khaled, K. Mishchenko, and P. Richtárik, “Tighter Theory for Local SGD on Identical and Heterogeneous Data,”
Original
1909
Earlier work this paper cites.
Y. Jiang, J. Konečný, K. Rush, and S. Kannan, “Improving Federated Learning Personalization via Model Agnostic Meta Learning,”
Original
1909
Earlier work this paper cites.
A. Khaled, K. Mishchenko, and P. Richtárik, “First Analysis of Local GD on Heterogeneous Data,”
Original
1909
Earlier work this paper cites.
S. P. Karimireddy
Original
1910
Earlier work this paper cites.
D. Li and J. Wang, “FedMD: Heterogenous Federated Learning via Model Distillation,”
Original
1910
Earlier work this paper cites.
F. Haddadpour and M. Mahdavi, “On the Convergence of Local Descent Methods in Federated Learning,”
Original
1910
Earlier work this paper cites.
X. Li, W. Yang, S. Wang, and Z. Zhang, “Communication-Efficient Local Decentralized SGD Methods,”
Original
1910
Earlier work this paper cites.

Similar

K. Pillutla, S. M. Kakade, and Z. Harchaoui, “Robust Aggregation for Federated Learning,”
Original
1912
Cited alongside, same era.
M. G. Arivazhagan, V. Aggarwal, A. K. Singh, and S. Choudhary, “Federated Learning with Personalization Layers,”
Original
1912
Cited alongside, same era.
J.-J. Moreau, “Propriétés des applications ‘prox’,”
1963
Cited alongside, same era.
C. Lemaréchal and C. Sagastizábal, “Practical Aspects of the Moreau–Yosida Regularization: Theoretical Preliminaries,”
1997
Cited alongside, same era.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-Based Learning Applied to Document Recognition,”
1998
Cited alongside, same era.

Then

V. Smith, C.-K. Chiang, M. Sanjabi, and A. Talwalkar, “Federated Multi-Task Learning,”
Original
2018
Later among the works it cites.
H. B. McMahan, D. Ramage, K. Talwar, and L. Zhang, “Learning Differentially Private Recurrent Language Models,”
Original
2018
Later among the works it cites.
A. Nichol, J. Achiam, and J. Schulman, “On First-Order Meta-Learning Algorithms,”
Original
2018
Later among the works it cites.
F. Chen, Z. Dong, Z. Li, and X. He, “Federated Meta-Learning for Recommendation,” Feb. 2018
2018
Later among the works it cites.
H. Lin, J. Mairal, and Z. Harchaoui, “Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice,”
Original
2018

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

A. Fallah, A. Mokhtari, and A. Ozdaglar, “Personalized Federated Learning: A Meta-Learning Approach,”

Original

2002

Cited alongside, same era.

F. Hanzely and P. Richtárik, “Federated Learning of a Mixture of Global and Local Models,”

Original

2002

Cited alongside, same era.

Y. Mansour, M. Mohri, J. Ro, and A. T. Suresh, “Three Approaches for Personalization with Applications to Federated Learning,”

Original

2002

Cited alongside, same era.

Y. Deng, M. M. Kamani, and M. Mahdavi, “Adaptive Personalized Federated Learning,”

Original

2003

Cited alongside, same era.

S. Reddi

Original

2003

Cited alongside, same era.

J. C. Duchi, M. I. Jordan, and M. J. Wainwright, “Privacy Aware Learning,”

2014

Cited alongside, same era.

S. Bubeck, “Convex Optimization: Algorithms and Complexity,”

Original

2015

Cited alongside, same era.