Fetching the paper…

Reading the bibliography…

2015

SDCA without Duality

Shalev-Shwartz, Shai

Original
PDF
Post on X

Understand

Stochastic Dual Coordinate Ascent is a popular method for solving regularized loss minimization for the case of convex losses.

In this paper we show how a variant of SDCA can be applied for non-convex losses.
We prove linear convergence rate even if individual loss functions are non-convex as long as the expected loss is convex.

SDCA without Duality · Around

Built on

A stochastic gradient method with an exponential convergence rate for finite training sets
Nicolas Le Roux, Mark Schmidt, and Francis Bach · 2012
Earlier work this paper cites.
Accelerating stochastic gradient descent using predictive variance reduction
Rie Johnson and Tong Zhang · 2013
Earlier work this paper cites.

Similar

Semi-stochastic gradient descent methods
Original
Jakub Konečnỳ and Peter Richtárik · 2013
Cited alongside, same era.
Stochastic dual coordinate ascent methods for regularized loss minimization
Shai Shalev-Shwartz and Tong Zhang · 2013
Cited alongside, same era.
Saga: A fast incremental gradient method with support for non-strongly convex composite objectives
Aaron Defazio, Francis Bach, and Simon Lacoste-Julien
Cited in the paper.
Finito: A faster, permutable incremental gradient method for big data problems
Original
Aaron J Defazio, Tibério S Caetano, and Justin Domke
Cited in the paper.

Then

Understanding Machine Learning: From Theory to Algorithms
Shai Shalev-Shwartz and Shai Ben-David · 2014
Later among the works it cites.
Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization
S. Shalev-Shwartz and T. Zhang · 2015
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…