Variable Metric Stochastic Approximation Theory

A stochastic approximation method

H. E. Robbins and S. Monro · 1951

Earlier work this paper cites.

Multidimensional stochastic approximation methods

J. R. Blum · 1954

Earlier work this paper cites.

A convergence theorem for non negative almost supermartingales and some applications

H. E. Robbins and D. O. Siegmund · 1971

Earlier work this paper cites.

Methods of Information Geometry

S. Amari and H. Nagaoka · 1993

Earlier work this paper cites.

Exponentiated gradient versus gradient descent for linear predictors

J. Kivinen and M. K. Warmuth · 1997

Earlier work this paper cites.

Natural gradient works efficiently in learning

S. Amari · 1998

Earlier work this paper cites.

A statistical study of on-line learning

N. Murata · 1998

Cited alongside, same era.

Online algorithms and stochastic approximations

L. Bottou · 1998

Cited alongside, same era.

Numerical Optimization

J. Nocedal and S. J. Wright · 1999

Cited alongside, same era.

Linear Estimation

Original

T. Kailath, A. Sayed, and B. Hassibi · 2000

Cited alongside, same era.

Relative loss bounds for on-line density estimation with the exponential family of distributions

K. Azoury and M. K. Warmuth · 2001

Cited alongside, same era.

Fast curvature matrix-vector products for second-order gradient descent

N. N. Schraudolph · 2002

Cited alongside, same era.

Online convex programming and generalised infinitesimal gradient ascent

M. Zinkevich · 2003

Later among the works it cites.

On-line learning for very large datasets

L. Bottou and Y. LeCun · 2005

Later among the works it cites.

A stochastic quasi-Newton method for online convex optimization

N. N. Schraudolph, J. Yu, and S. Günter · 2007

Later among the works it cites.

Pegasos: Primal estimated sub-gradient solver for SVM

S. Shalev-Shwartz, Y. Singer, and N. Srebro · 2007

Later among the works it cites.

Logarithmic regret algorithms for online convex optimization

E. Hazan, A. Agarwal, and S. Kale · 2007

Later among the works it cites.