Fetching the paper…

Reading the bibliography…

2015

Efficient Elastic Net Regularization for Sparse Linear Models

Lipton, Zachary C., Elkan, Charles

Original
PDF
Post on X

Understand

This paper presents an algorithm for efficient training of sparse linear models with elastic net regularization.

Extending previous work on delayed updates, the new algorithm applies stochastic gradient updates to non-zero features only, bringing weights current as needed with closed-form updates.
Closed-form delayed updates for the $\ell_1$, $\ell_{\infty}$, and rarely used $\ell_2$ regularizers have been described previously.
This paper provides closed-form updates for the popular squared norm $\ell^2_2$ and elastic net regularizers.

Efficient Elastic Net Regularization for Sparse Linear Models · Around

Built on

Regularization and variable selection via the elastic net
Original
Hui Zou and Trevor Hastie · 2005
Earlier work this paper cites.
Lazy sparse stochastic gradient descent for regularized multinomial logistic regression
Bob Carpenter · 2008
Earlier work this paper cites.
Efficient projections onto the l 1-ball for learning in high dimensions
John Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra · 2008
Earlier work this paper cites.
Sparse online learning via truncated gradient
John Langford, Lihong Li, and Tong Zhang · 2009
Earlier work this paper cites.
Efficient learning using forward-backward splitting
Yoram Singer and John C Duchi · 2009
Earlier work this paper cites.

Similar

Adaptive subgradient methods for online learning and stochastic optimization
John Duchi, Elad Hazan, and Yoram Singer · 2011
Cited alongside, same era.
Stochastic gradient descent tricks
Léon Bottou · 2012
Cited alongside, same era.
Adadelta: an adaptive learning rate method
Original
Matthew D Zeiler · 2012
Cited alongside, same era.
Accelerating stochastic gradient descent using predictive variance reduction
Rie Johnson and Tong Zhang · 2013
Cited alongside, same era.

Then

Proximal algorithms
Neal Parikh and Stephen Boyd · 2013
Later among the works it cites.
Minimizing finite sums with the stochastic average gradient
Original
Mark Schmidt, Nicolas Le Roux, and Francis Bach · 2013
Later among the works it cites.
Efficient mini-batch training for stochastic optimization
Mu Li, Tong Zhang, Yuqiang Chen, and Alexander J Smola · 2014
Later among the works it cites.
Optimal thresholding of classifiers to maximize f1 measure
Zachary C Lipton, Charles Elkan, and Balakrishnan Naryanaswamy · 2014
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…