Direct Policy Gradients: Direct Optimization of Policies in Discrete Action Spaces

Understand

Direct optimization is an appealing framework that replaces integration with optimization of a random objective for approximating gradients in models with discrete random variables.

• A$^\star$ sampling is a framework for optimizing such random objectives over large spaces.
• We show how to combine these techniques to yield a reinforcement learning algorithm that approximates a policy gradient by finding trajectories that optimize a random objective.
• We call the resulting algorithms "direct policy gradient" (DirPG) algorithms.