Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attacks

Understand

Recent advances in large-scale distributed learning algorithms have enabled communication-efficient training via SignSGD.

• Unfortunately, a major issue continues to plague distributed learning: namely, Byzantine failures may incur serious degradation in learning accuracy.
• This paper proposes Election Coding, a coding-theoretic framework to guarantee Byzantine-robustness for SignSGD with Majority Vote, which uses minimum worker-master communication in both directions.
• The suggested framework explores new information-theoretic limits of finding the majority opinion when some workers could be malicious, and paves the road to implement robust and efficient distributed learning algorithms.