Self-tuned mirror descent schemes for smooth and nonsmooth high-dimensional stochastic optimization

Understand

We consider randomized block coordinate stochastic mirror descent (RBSMD) methods for solving high-dimensional stochastic optimization problems with strongly convex objective functions.

• Our goal is to develop RBSMD schemes that achieve a rate of convergence with a minimum constant factor with respect to the choice of the stepsize sequence.
• To this end, we consider both subgradient and gradient RBSMD methods addressing nonsmooth and smooth problems, respectively.
• For each scheme, (i) we develop self-tuned stepsize rules characterized in terms of problem parameters and algorithm settings; (ii) we show that the non-averaging iterate generated by the underlying RBSMD method converges to the optimal solution both in an almost sure and a mean sense; (iii) we show that the mean squared error is minimized.