Acceleration Methods

Understand

This monograph covers some recent advances in a range of acceleration techniques frequently used in convex optimization.

• We first use quadratic optimization problems to introduce two key families of methods, namely momentum and nested optimization schemes.
• They coincide in the quadratic case to form the Chebyshev method.
• We discuss momentum methods in detail, starting with the seminal work of Nesterov and structure convergence proofs using a few master templates, such as that for optimized gradient methods, which provide the key benefit of showing how momentum methods optimize convergence guarantees.