Solving Non-Rectangular Reward-Robust MDPs via Frequency Regularization

Understand

In robust Markov decision processes (RMDPs), it is assumed that the reward and the transition dynamics lie in a given uncertainty set.

• By targeting maximal return under the most adversarial model from that set, RMDPs address performance sensitivity to misspecified environments.
• Yet, to preserve computational tractability, the uncertainty set is traditionally independently structured for each state.
• This so-called rectangularity condition is solely motivated by computational concerns.