Policy Optimization over General State and Action Spaces

Understand

Reinforcement learning (RL) problems over general state and action spaces are notoriously challenging.

• In contrast to the tableau setting, one can not enumerate all the states and then iteratively update the policies for each state.
• This prevents the application of many well-studied RL methods especially those with provable convergence guarantees.
• In this paper, we first present a substantial generalization of the recently developed policy mirror descent method to deal with general state and action spaces.