2018

On Landscape of Lagrangian Functions and Stochastic Search for Constrained Nonconvex Optimization

Chen, Zhehui, Li, Xingguo, Yang, Lin F. et al.

Understand

We study constrained nonconvex optimization problems in machine learning, signal processing, and stochastic control.

It is well-known that these problems can be rewritten to a minimax problem in a Lagrangian form.
However, due to the lack of convexity, their landscape is not well understood and how to find the stable equilibria of the Lagrangian function is still unknown.
To bridge the gap, we study the landscape of the Lagrangian function.

The brownian movement and stochastic equations
Doob, J. L · 1942
Earlier work this paper cites.
Gradient methods for minimizing functionals
Polyak, B. T · 1963
Earlier work this paper cites.
Linear and nonlinear programming
Luenberger, D. G · 1984
Earlier work this paper cites.
Optimal unsupervised learning in a single-layer linear feedforward neural network
Sanger, T. D · 1989
Earlier work this paper cites.
Error bounds and convergence analysis of feasible descent methods: a general approach
Luo, Z.-Q · 1993
Earlier work this paper cites.
Stochastic approximation and recursive algorithm and applications
Harold, J · 1997
Earlier work this paper cites.
Fisher discriminant analysis with kernels
Mika, S · 1999
Earlier work this paper cites.
Policy gradient methods for reinforcement learning with function approximation
Sutton, R. S · 2000
Earlier work this paper cites.
The elements of statistical learning
Friedman, J · 2001
Earlier work this paper cites.
Stochastic approximation and recursive algorithms and applications
Kushner, H · 2003
Earlier work this paper cites.

Convex optimization
Boyd, S · 2004
Cited alongside, same era.
Abstract algebra
Dummit, D. S · 2004
Cited alongside, same era.
Sufficient dimension reduction via inverse regression: A minimum discrepancy approach
Cook, R. D · 2005
Cited alongside, same era.
Generalized hebbian algorithm for incremental singular value decomposition in natural language processing
Gorrell, G · 2006
Cited alongside, same era.
Markov processes: characterization and convergence
Ethier, S. N · 2009
Cited alongside, same era.
Lectures on stochastic programming: modeling and theory
Shapiro, A · 2009
Cited alongside, same era.
Primal-dual first-order methods with { \{ \ \backslash mathcal { \{ O } \} (1/ \ \backslash epsilon) } \} iteration-complexity for cone programming
Lan, G · 2011
Cited alongside, same era.
On multi-parameter semimartingales, their integrals and weak convergence
Nowakowski, B. D · 2013
Cited alongside, same era.
Optimal primal-dual methods for a class of saddle point problems
Chen, Y · 2014
Cited alongside, same era.
Generative adversarial nets
Goodfellow, I · 2014
Cited alongside, same era.
Efficient algorithms for large-scale generalized eigenvector computation and canonical correlation analysis
Ge, R
Cited in the paper.
Matrix completion has no spurious local minimum
Ge, R
Cited in the paper.

Primal-dual subgradient methods for minimizing uniformly convex functions
Original
Iouditski, A · 2014
Later among the works it cites.
Escaping from saddle points—online stochastic gradient for tensor decomposition
Ge, R · 2015
Later among the works it cites.
Doubly accelerated methods for faster CCA and generalized eigendecomposition
Original
Allen-Zhu, Z · 2016
Later among the works it cites.
Global optimality of local search for low rank matrix recovery
Original
Bhojanapalli, S · 2016
Later among the works it cites.
Riemannian preconditioning
Mishra, B · 2016
Later among the works it cites.
A geometric analysis of phase retrieval
Sun, J · 2016
Later among the works it cites.
Stochastic approximation for canonical correlation analysis
Arora, R · 2017
Later among the works it cites.
Online multiview representation learning: Dropping convexity for better efficiency
Original
Chen, Z · 2017
Later among the works it cites.
The global optimization geometry of nonsymmetric matrix factorization and sensing
Original
Zhu, Z · 2017
Later among the works it cites.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…