Distributed Gradient Descent: Nonconvergence to Saddle Points and the Stable-Manifold Theorem

Understand

The paper studies a distributed gradient descent (DGD) process and considers the problem of showing that in nonconvex optimization problems, DGD typically converges to local minima rather than saddle points.

• The paper considers unconstrained minimization of a smooth objective function.
• In centralized settings, the problem of demonstrating nonconvergence to saddle points of gradient descent (and variants) is typically handled by way of the stable-manifold theorem from classical dynamical systems theory.
• However, the classical stable-manifold theorem is not applicable in distributed settings.