Understanding Nesterov's Acceleration via Proximal Point Method

Understand

The proximal point method (PPM) is a fundamental method in optimization that is often used as a building block for designing optimization algorithms.

• In this work, we use the PPM method to provide conceptually simple derivations along with convergence analyses of different versions of Nesterov's accelerated gradient method (AGM).
• The key observation is that AGM is a simple approximation of PPM, which results in an elementary derivation of the update equations and stepsizes of AGM.
• This view also leads to a transparent and conceptually simple analysis of AGM's convergence by using the analysis of PPM.