Decentralized Deep Learning using Momentum-Accelerated Consensus

Understand

We consider the problem of decentralized deep learning where multiple agents collaborate to learn from a distributed dataset.

• While there exist several decentralized deep learning approaches, the majority consider a central parameter-server topology for aggregating the model parameters from the agents.
• However, such a topology may be inapplicable in networked systems such as ad-hoc mobile networks, field robotics, and power network systems where direct communication with the central parameter server may be inefficient.
• In this context, we propose and analyze a novel decentralized deep learning algorithm where the agents interact over a fixed communication topology (without a central server).