First-Order Methods for Wasserstein Distributionally Robust MDP

Understand

Markov decision processes (MDPs) are known to be sensitive to parameter specification.

• Distributionally robust MDPs alleviate this issue by allowing for \emph{ambiguity sets} which give a set of possible distributions over parameter sets.
• The goal is to find an optimal policy with respect to the worst-case parameter distribution.
• We propose a framework for solving Distributionally robust MDPs via first-order methods, and instantiate it for several types of Wasserstein ambiguity sets.