RORL: Robust Offline Reinforcement Learning via Conservative Smoothing

Understand

Offline reinforcement learning (RL) provides a promising direction to exploit massive amount of offline data for complex decision-making tasks.

• Due to the distribution shift issue, current offline RL algorithms are generally designed to be conservative in value estimation and action selection.
• However, such conservatism can impair the robustness of learned policies when encountering observation deviation under realistic conditions, such as sensor errors and adversarial attacks.
• To trade off robustness and conservatism, we propose Robust Offline Reinforcement Learning (RORL) with a novel conservative smoothing technique.