FedStale: leveraging stale client updates in federated learning

Understand

Federated learning algorithms, such as FedAvg, are negatively affected by data heterogeneity and partial client participation.

• To mitigate the latter problem, global variance reduction methods, like FedVARP, leverage stale model updates for non-participating clients.
• These methods are effective under homogeneous client participation.
• Yet, this paper shows that, when some clients participate much less than others, aggregating updates with different levels of staleness can detrimentally affect the training process.