On the convex formulations of robust Markov decision processes

Understand

Robust Markov decision processes (MDPs) are used for applications of dynamic optimization in uncertain environments and have been studied extensively.

• Many of the main properties and algorithms of MDPs, such as value iteration and policy iteration, extend directly to RMDPs.
• Surprisingly, there is no known analog of the MDP convex optimization formulation for solving RMDPs.
• This work describes the first convex optimization formulation of RMDPs under the classical sa-rectangularity and s-rectangularity assumptions.