Distributed Gradient Methods for Nonconvex Optimization: Local and Global Convergence Guarantees

——, “Annealing for distributed global optimization,” 2019, to appear in Proceedings of IEEE Conference on Decision and Control

Original

1903

Earlier work this paper cites.

X. Chen, S. Du, and X. Tong, “On stationary-point hitting time and ergodicity of stochastic gradient Langevin dynamics,” 2019, online: https://arxiv.org/pdf/1904:13016.pdf

1904

Earlier work this paper cites.

S. Vlaski and A. H. Sayed, “Distributed learning in non-convex environments–Part I: Agreement at a linear rate,” 2019, arXiv preprint arXiv:1907.01848

Original

1907

Earlier work this paper cites.

——, “Distributed learning in non-convex environments–Part II: Polynomial escape from saddle-points,” 2019, arXiv preprint arXiv:1907.01849

Original

1907

Earlier work this paper cites.

B. Swenson, S. Kar, H. V. Poor, and J. M. F. Moura, “Distributed global optimization by annealing,” 2019, to appear in Proceedings of IEEE International Workshop on Computational Advances in Multisensor Adaptive Processing (CAMSAP), arXiv preprint arXiv:1907.08802

Original

1907

Earlier work this paper cites.

B. Swenson, A. Sridhar, and H. V. Poor, “On distributed stochastic gradient algorithms for global optimization,” 2020, to appear in Proceedings of International Conference on Acoustics, Speach, and Signal Processing, arXiv preprint arXiv:1910.09587

Original

1910

Earlier work this paper cites.

E. A. Coddington and N. Levinson, Theory of Ordinary Differential Equations . Tata McGraw-Hill, 1955

1955

Earlier work this paper cites.

C.-R. Hwang et al. , “Laplace’s method revisited: Weak convergence of probability measures,” The Annals of Probability , vol. 8, no. 6, pp. 1177–1182, 1980

1980

Earlier work this paper cites.

S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing,” Science , vol. 220, no. 4598, pp. 671–680, 1983

1983

Earlier work this paper cites.

H. J. Kushner, “Asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects: Global minimization via Monte Carlo,” SIAM Journal on Applied Mathematics , vol. 47, no. 1, pp. 169–185, 1987

1987

Earlier work this paper cites.

T.-S. Chiang, C.-R. Hwang, and S. J. Sheu, “Diffusion for global optimization in ℝ n \mathbb{R}^{n} ,” SIAM Journal on Control and Optimization , vol. 25, no. 3, pp. 737–753, 1987

1987

Earlier work this paper cites.

R. Pemantle, “Nonconvergence to unstable points in urn models and stochastic approximations,” The Annals of Probability , vol. 18, no. 2, pp. 698–712, 1990

1990

Earlier work this paper cites.

S. B. Gelfand and S. K. Mitter, “Recursive stochastic algorithms for global optimization in ℝ d \mathbb{R}^{d} ,” SIAM Journal on Control and Optimization , vol. 29, no. 5, pp. 999–1018, 1991

1991

Earlier work this paper cites.

G. Yin, “Rates of convergence for a class of global stochastic optimization algorithms,” SIAM Journal on Optimization , vol. 10, no. 1, pp. 99–120, 1999

1999

Earlier work this paper cites.

M. Benaïm, “Dynamics of stochastic approximation algorithms,” in Seminaire de Probabilites 33 . Springer, 1999, pp. 1–68

1999

Earlier work this paper cites.

B. Swenson, R. Murray, H. V. Poor, and S. Kar, “Distributed stochastic gradient descent: Nonconvexity, nonsmoothness, and convergence to local minima,” 2020, arXiv preprint arXiv:2003.02818

Original

2003

Cited alongside, same era.

M. Rabbat and R. Nowak, “Distributed optimization in sensor networks,” in Proceedings of the 3rd international symposium on information processing in sensor networks , 2004, pp. 20–27

2004

Cited alongside, same era.

R. Olfati-Saber, J. A. Fax, and R. M. Murray, “Consensus and cooperation in networked multi-agent systems,” Proceedings of the IEEE , vol. 95, no. 1, pp. 215–233, 2007

2007

Cited alongside, same era.

B. Swenson, R. Murray, H. V. Poor, and S. Kar, “Distributed gradient flow: Nonsmoothness, nonconvexity, and saddle point evasion,” 2020, arXiv preprint arXiv:2008.05387

Original

2008

Cited alongside, same era.

K. Kawaguchi, “Deep learning without poor local minima,” in Proceedings of Advances in Neural Information Processing Systems , 2016, pp. 586–594

2016

Later among the works it cites.

R. Ge, J. D. Lee, and T. Ma, “Matrix completion has no spurious local minimum,” in Advances in Neural Information Processing Systems , 2016, pp. 2973–2981

2016

Later among the works it cites.

J. D. Lee, M. Simchowitz, M. I. Jordan, and B. Recht, “Gradient descent only converges to minimizers,” in Conference on Learning Theory, PMLR , 2016, pp. 1246–1257

2016

Later among the works it cites.

A. Wibisono, A. C. Wilson, and M. I. Jordan, “A variational perspective on accelerated methods in optimization,” Proceedings of the National Academy of Sciences , vol. 113, no. 47, pp. E7351–E7358, 2016

2016

Later among the works it cites.