Efficient Policy Iteration for Robust Markov Decision Processes via Regularization

Understand

Robust Markov decision processes (MDPs) provide a general framework to model decision problems where the system dynamics are changing or only partially known.

• Efficient methods for some \texttt{sa}-rectangular robust MDPs exist, using its equivalence with reward regularized MDPs, generalizable to online settings.
• In comparison to \texttt{sa}-rectangular robust MDPs, \texttt{s}-rectangular robust MDPs are less restrictive but much more difficult to deal with.
• Interestingly, recent works have established the equivalence between \texttt{s}-rectangular robust MDPs and policy regularized MDPs.