Asynchronous Byzantine Machine Learning (the case of SGD)

Understand

Asynchronous distributed machine learning solutions have proven very effective so far, but always assuming perfectly functioning workers.

• In practice, some of the workers can however exhibit Byzantine behavior, caused by hardware failures, software bugs, corrupt data, or even malicious attacks.
• We introduce \emph{Kardam}, the first distributed asynchronous stochastic gradient descent (SGD) algorithm that copes with Byzantine workers.
• Kardam consists of two complementary components: a filtering and a dampening component.