A Minimalist Approach to Offline Reinforcement Learning

Understand

Offline reinforcement learning (RL) defines the task of learning from a fixed batch of data.

• Due to errors in value estimation from out-of-distribution actions, most offline RL algorithms take the approach of constraining or regularizing the policy with the actions contained in the dataset.
• Built on pre-existing RL algorithms, modifications to make an RL algorithm work offline comes at the cost of additional complexity.
• Offline RL algorithms introduce new hyperparameters and often leverage secondary components such as generative models, while adjusting the underlying RL algorithm.