Accuracy of Discretely Sampled Stochastic Policies in Continuous-time Reinforcement Learning

Understand

Stochastic policies (also known as relaxed controls) are widely used in continuous-time reinforcement learning algorithms.

• However, executing a stochastic policy and evaluating its performance in a continuous-time environment remain open challenges.
• This work introduces and rigorously analyzes a policy execution framework that samples actions from a stochastic policy at discrete time points and implements them as piecewise constant controls.
• We prove that as the sampling mesh size tends to zero, the controlled state process converges weakly to the dynamics with coefficients aggregated according to the stochastic policy.