Local Adaptivity in Federated Learning: Convergence and Consistency

Understand

The federated learning (FL) framework trains a machine learning model using decentralized data stored at edge client devices by periodically aggregating locally trained models.

• Popular optimization algorithms of FL use vanilla (stochastic) gradient descent for both local updates at clients and global updates at the aggregating server.
• Recently, adaptive optimization methods such as AdaGrad have been studied for server updates.
• However, the effect of using adaptive optimization methods for local updates at clients is not yet understood.