Robust Reinforcement Learning using Offline Data

Understand

The goal of robust reinforcement learning (RL) is to learn a policy that is robust against the uncertainty in model parameters.

• Parameter uncertainty commonly occurs in many real-world RL applications due to simulator modeling errors, changes in the real-world system dynamics over time, and adversarial disturbances.
• Robust RL is typically formulated as a max-min problem, where the objective is to learn the policy that maximizes the value against the worst possible models that lie in an uncertainty set.
• In this work, we propose a robust RL algorithm called Robust Fitted Q-Iteration (RFQI), which uses only an offline dataset to learn the optimal robust policy.