Pathwise coordinate optimization

Understand

We consider ``one-at-a-time'' coordinate-wise descent algorithms for a class of convex optimization problems.

• An algorithm of this kind has been proposed for the $L_1$-penalized regression (lasso) in the literature, but it seems to have been largely ignored.
• Indeed, it seems that coordinate-wise algorithms are not often used in convex optimization.
• We show that this algorithm is very competitive with the well-known LARS (or homotopy) procedure in large lasso problems, and that it can be applied to related methods such as the garotte and elastic net.