Understand
Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance.
- The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data.
- The parameter updates occur in continuous time and satisfy a stochastic differential equation.
- This paper analyzes the asymptotic convergence rate of the SGDCT algorithm by proving a central limit theorem (CLT) for strongly convex objective functions and, under slightly stronger conditions, for non-convex objective functions as well.
Built on
D. J. Sakrison, A continuous Kiefer-Wolfowitz procedure for random processes, Ann. Math. Statist., Vol. 35, No. 2, (1964), pp. 590-599
1964
Earlier work this paper cites.
N. Ikeda and S. Watanabe, A comparison theorem for solutions of stochastic differential equations and its applications, Osaka J. Math. , Vol. 14, (1977), pp. 619-633
1977
Earlier work this paper cites.
I. Basawa and B. Rao, Asymptotic inference for stochastic processes, Stochastic Processes and their Applications , Vol.10, No. 3, (1980), pp. 221-254
1980
Earlier work this paper cites.
A. Benveniste, M. Metivier, and P. Priouret, Adaptive Algorithms and Stochastic Approximations. Springer-Verlag , 1990
1990
Earlier work this paper cites.
1993
Earlier work this paper cites.
A. Yu. Veretennikov, On polynomial mixing bounds for stochastic differential equations, Stochastic Processes and their Applications Vol. 70, Issue 1, (1997), pp. 115-127
1997
Earlier work this paper cites.
Similar
B. L. S. P. Rao, Statistical inference for diffusion type processes, Arnold, 1999
1999
Cited alongside, same era.
K. Doya, Reinforcement learning in continuous time and space. Neural computation, Vol. 12, No. 1, pp. 219-245, 2000
2000
Cited alongside, same era.
O. Elerian, S. Chib, and N. Shephard, Likelihood inference for discretely observed nonlinear diffusions, Econometrica
2001
Cited alongside, same era.
E. Pardoux and A.Yu. Veretennikov, On Poisson equation and diffusion approximation 1, Annals of Probability
2001
Cited alongside, same era.
E. Pardoux and A. Y. Veretennikov, On Poisson equation and diffusion approximation 2, The Annals of Probability, Vol. 31, No. 3, (2003), pp. 1166-1192
2003
Cited alongside, same era.
Y. Kutoyants, Statistical Inference for Ergodic Diffusion Processes. Springer , 2004
2004
Cited alongside, same era.
Then
C. Chicone, Ordinary differential equations with applications, second edition, Springer, New York, 2006
2006
Later among the works it cites.
J. P. N. Bishwal, Parameter estimation in stochastic differential equations, in: Lecture Notes in Mathematics, Vol. 1923, Springer Science & Business Media, 2008
2008
Later among the works it cites.
M. Raginsky and J. Bouvrie, Continuous-time stochastic mirror descent on a network: variance reduction, consensus, convergence, IEEE Conference on Decision and Control , 2012
2012
Later among the works it cites.
2016
Later among the works it cites.
J. Sirignano and K. Spiliopoulos, Stochastic Gradient Descent in Continuous Time, SIAM Journal on Financial Mathematics , Vol. 8, Issue 1, (2017), pp. 933–961
2017
Closest in time.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…