On the Convergence and Sample Efficiency of Variance-Reduced Policy Gradient Method

Understand

Policy gradient (PG) gives rise to a rich class of reinforcement learning (RL) methods.

• Recently, there has been an emerging trend to accelerate the existing PG methods such as REINFORCE by the \emph{variance reduction} techniques.
• However, all existing variance-reduced PG methods heavily rely on an uncheckable importance weight assumption made for every single iteration of the algorithms.
• In this paper, a simple gradient truncation mechanism is proposed to address this issue.