Holdout SGD: Byzantine Tolerant Federated Learning

Understand

This work presents a new distributed Byzantine tolerant federated learning algorithm, HoldOut SGD, for Stochastic Gradient Descent (SGD) optimization.

• HoldOut SGD uses the well known machine learning technique of holdout estimation, in a distributed fashion, in order to select parameter updates that are likely to lead to models with low loss values.
• This makes it more effective at discarding Byzantine workers inputs than existing methods that eliminate outliers in the parameter-space of the learned model.
• HoldOut SGD first randomly selects a set of workers that use their private data in order to propose gradient updates.