Gradient-free optimization of highly smooth functions: improved analysis and a new algorithm

Understand

This work studies minimization problems with zero-order noisy oracle information under the assumption that the objective function is highly smooth and possibly satisfies additional properties.

• We consider two kinds of zero-order projected gradient descent algorithms, which differ in the form of the gradient estimator.
• The first algorithm uses a gradient estimator based on randomization over the $\ell_2$ sphere due to Bach and Perchet (2016).
• We present an improved analysis of this algorithm on the class of highly smooth and strongly convex functions studied in the prior work, and we derive rates of convergence for two more general classes of non-convex functions.