Recent Theoretical Advances in Non-Convex Optimization

Understand

Motivated by recent increased interest in optimization algorithms for non-convex optimization in application to training deep neural networks and other optimization problems in data analysis, we give an overview of recent theoretical results on global performance guarantees of optimization algorithms for non-convex optimization.

• We start with classical arguments showing that general non-convex problems could not be solved efficiently in a reasonable time.
• Then we give a list of problems that can be solved efficiently to find the global minimizer by exploiting the structure of the problem as much as it is possible.
• Another way to deal with non-convexity is to relax the goal from finding the global minimum to finding a stationary point or a local minimum.