Simple and optimal methods for stochastic variational inequalities, II: Markovian noise and policy evaluation in reinforcement learning

Learning to predict by the methods of temporal differences

R. S. Sutton · 1988

Earlier work this paper cites.

Adaptive Algorithms and Stochastic Approximations

A. Benveniste, P. Priouret, and M. Métivier · 1990

Earlier work this paper cites.

Information-based complexity of linear operator equations

A.S. Nemirovski · 1992

Earlier work this paper cites.

Convergence rates for Markov chains

J. S. Rosenthal · 1995

Earlier work this paper cites.

Stochastic optimal control: the discrete-time case

D. P. Bertsekas and S. Shreve · 1996

Earlier work this paper cites.

An analysis of temporal-difference learning with function approximation

J. N. Tsitsiklis and B. Van Roy · 1997

Earlier work this paper cites.

Actor-critic algorithms

V. R. Konda and J. N. Tsitsiklis · 2000

Earlier work this paper cites.

Stochastic Approximation and Recursive Algorithms and Applications

H. J. Kushner and G. Yin · 2003

Earlier work this paper cites.

Least-squares policy iteration

M. G. Lagoudakis and R. Parr · 2003

Earlier work this paper cites.

Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control

J.C. Spall · 2003

Earlier work this paper cites.

Projected equations, variational inequalities, and temporal difference methods

D. P. Bertsekas · 2009

Cited alongside, same era.

Markov Chains and Stochastic Stability

S. Meyn, R. L. Tweedie, and P. W. Glynn · 2009

Cited alongside, same era.

Fast gradient-descent methods for temporal-difference learning with linear function approximation

R. S. Sutton, H. R. Maei, D. Precup, S. Bhatnagar, D. Silver, C. Szepesvári, and E. Wiewiora · 2009

Cited alongside, same era.

Ergodic mirror descent

J. C. Duchi, A. Agarwal, M. Johansson, and M. I. Jordan · 2012

Cited alongside, same era.

Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization I: A generic algorithmic framework

S. Ghadimi and G. Lan · 2012

Cited alongside, same era.

Policy evaluation with temporal differences: A survey and comparison

C. Dann, G. Neumann, and J. Peters · 2014

A finite time analysis of temporal difference learning with linear function approximation

J. Bhandari, D. Russo, and R. Singal · 2018

Later among the works it cites.

Linear stochastic approximation: How far does constant step-size and iterate averaging go?

C. Lakshminarayanan and C. Szepesvari · 2018

Later among the works it cites.

Reinforcement Learning: An Introduction

Richard S. Sutton and Andrew G. Barto · 2018

Later among the works it cites.

Mixing time estimation in reversible Markov chains from a single sample path

D. Hsu, A. Kontorovich, D. A. Levin, Y. Peres, C. Szepesvári, and G. Wolfer · 2019

Later among the works it cites.

Estimating the mixing time of ergodic Markov chains

Original

G. Wolfer and A. Kontorovich · 2019

Later among the works it cites.