S-NEAR-DGD: A Flexible Distributed Stochastic Gradient Method for Inexact Communication

Understand

We present and analyze a stochastic distributed method (S-NEAR-DGD) that can tolerate inexact computation and inaccurate information exchange to alleviate the problems of costly gradient evaluations and bandwidth-limited communication in large-scale systems.

• Our method is based on a class of flexible, distributed first order algorithms that allow for the trade-off of computation and communication to best accommodate the application setting.
• We assume that all the information exchange between nodes is subject to random distortion and that only stochastic approximations of the true gradients are available.
• Our theoretical results prove that the proposed algorithm converges linearly in expectation to a neighborhood of the optimal solution for strongly convex objective functions with Lipschitz gradients.