Generalized Federated Learning via Sharpness Aware Minimization

Understand

Federated Learning (FL) is a promising framework for performing privacy-preserving, distributed learning with a set of clients.

• However, the data distribution among clients often exhibits non-IID, i.e., distribution shift, which makes efficient optimization difficult.
• To tackle this problem, many FL algorithms focus on mitigating the effects of data heterogeneity across clients by increasing the performance of the global model.
• However, almost all algorithms leverage Empirical Risk Minimization (ERM) to be the local optimizer, which is easy to make the global model fall into a sharp valley and increase a large deviation of parts of local clients.