Convergence of Proximal Policy Gradient Method for Problems with Control Dependent Diffusion Coefficients

Understand

We prove convergence of the proximal policy gradient method for a class of constrained stochastic control problems with control in both the drift and diffusion of the state process.

• The problem requires either the running or terminal cost to be strongly convex, but other terms may be non-convex.
• The inclusion of control-dependent diffusion introduces additional complexity in regularity analysis of the associated backward stochastic differential equation.
• We provide sufficient conditions under which the control iterates converge linearly to the optimal control, by deriving representations and estimates of solutions to the adjoint backward stochastic differential equations.