Concentration bounds for two time scale stochastic approximation

Alekseev, Vladimir M. “An estimate for the perturbations of the solutions of ordinary differential equations.” Westnik Moskov Unn. Ser

1961

Earlier work this paper cites.

Krasovskii, N. N., Stability of motion

1963

Earlier work this paper cites.

Brauer, Fred. “Perturbations of nonlinear systems of differential equations.” Journal of Mathematical Analysis and Applications

1966

Earlier work this paper cites.

Borkar, Vivek S. “Stochastic approximation with two time scales.” Systems & Control Letters

1997

Earlier work this paper cites.

Konda, Vijaymohan R., and Vivek S. Borkar, “Actor-critic-type learning algorithms for Markov decision processes”, SIAM Journal of Control and Optimization

1999

Earlier work this paper cites.

Konda, Vijaymohan R., and John T. Tsitsiklis, “On actor-critic algorithms”, SIAM Journal of Control and Optimization

2000

Cited alongside, same era.

Collins, E. J., and David S. Leslie, “Convergent multiple-timescales reinforcement learning algorithms for normal form games”, Annals of Applied Probability

2003

Cited alongside, same era.

Konda, Vijaymohan R., and John N. Tsitsiklis. “Convergence rate of linear two-time-scale stochastic approximation.” Annals of applied probability

2004

Cited alongside, same era.

Borkar, Vivek S. Stochastic approximation: a dynamical systems viewpoint

2008

Cited alongside, same era.

Liu, Q., and Watbled F. “Exponential inequalities for martingales and asymptotic properties of the free energy of directed polymers in a random environment.” Stochastic Processes and their Applications

2009

Cited alongside, same era.

Bhatnagar, S., Prasad, H. L., and Prashanth, L. A., Stochastic Recursive Algorithms for Optimization: Simultaneous Perturbation Methods

2013

Later among the works it cites.

Thoppe, Gugan, and Vivek S. Borkar. “A Concentration Bound for Stochastic Approximation via Alekseev’s Formula.” arXiv preprint arXiv:1506.08657

Original

2015

Later among the works it cites.

Dalal, Gal, Balazs Szorenyi, Gugan Thoppe, and Shie Mannor. “Concentration Bounds for Two Timescale Stochastic Approximation with Applications to Reinforcement Learning.” arXiv preprint

Original

2017

Later among the works it cites.

Borkar, Aseem V., Vivek S. Borkar and Arpita Sinha. “Aerial monitoring of slow moving convoys using elliptical orbits.” submitted

2018

Closest in time.

Kumar, Bhumesh, Vivek S. Borkar, and Akhil Shetty. ”Bounds for Tracking Error in Constant Stepsize Stochastic Approximation.” arXiv preprint arXiv:1802.07759