Variance Penalized On-Policy and Off-Policy Actor-Critic

Understand

Reinforcement learning algorithms are typically geared towards optimizing the expected return of an agent.

• However, in many practical applications, low variance in the return is desired to ensure the reliability of an algorithm.
• In this paper, we propose on-policy and off-policy actor-critic algorithms that optimize a performance criterion involving both mean and variance in the return.
• Previous work uses the second moment of return to estimate the variance indirectly.