Fetching the paper…

Reading the bibliography…

2014

Fast Distributed Coordinate Descent for Non-Strongly Convex Losses

Fercoq, Olivier, Qu, Zheng, Richtárik, Peter et al.

Original
PDF
Post on X

Understand

We propose an efficient distributed randomized coordinate descent method for minimizing regularized non-strongly convex loss functions.

The method attains the optimal $O(1/k^2)$ convergence rate, where $k$ is the iteration counter.
The core of the work is the theoretical study of stepsize parameters.
We have implemented the method on Archer - the largest supercomputer in the UK - and show that the method is capable of solving a (synthetic) LASSO optimization problem with 50 billion variables.

Fast Distributed Coordinate Descent for Non-Strongly Convex Losses · Around

Built on

“A method of solving a convex programming problem with convergence rate
Yurii Nesterov, · 1983
Earlier work this paper cites.
Numerical linear algebra
Grégoire Allaire and Sidi Mahmoud Kaber, · 2002
Earlier work this paper cites.
“On accelerated proximal gradient methods for convex-concave optimization,”
Paul Tseng, · 2008
Earlier work this paper cites.
“A fast iterative shrinkage-thresholding algorithm for linear inverse problems,”
Amir Beck and Marc Teboulle, · 2009
Earlier work this paper cites.

Similar

“Parallel coordinate descent for l1-regularized loss minimization,”
Joseph K. Bradley, Aapo Kyrola, Danny Bickson, and Carlos Guestrin, · 2011
Cited alongside, same era.
“Parallel coordinate descent methods for big data optimization,”
Original
Peter Richtárik and Martin Takáč, · 2012
Cited alongside, same era.
“Distributed coordinate descent method for learning with big data,”
Original
Peter Richtárik and Martin Takáč, · 2013
Cited alongside, same era.
“Accelerated, parallel and proximal coordinate descent,”
Original
Olivier Fercoq and Peter Richtárik, · 2013
Cited alongside, same era.

Then

“Smooth minimization of nonsmooth functions with parallel coordinate descent methods,”
Original
Olivier Fercoq and Peter Richtárik, · 2013
Later among the works it cites.
“Stochastic dual coordinate ascent methods for regularized loss,”
Shai Shalev-Shwartz and Tong Zhang, · 2013
Later among the works it cites.
“Mini-batch primal and dual methods for SVMs,”
Original
Martin Takáč, Avleen Bijral, Peter Richtárik, and Nathan Srebro, · 2013
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…