Model-Based Offline Planning

Understand

Offline learning is a key part of making reinforcement learning (RL) useable in real systems.

• Offline RL looks at scenarios where there is data from a system's operation, but no direct access to the system when learning a policy.
• Recent work on training RL policies from offline data has shown results both with model-free policies learned directly from the data, or with planning on top of learnt models of the data.
• Model-free policies tend to be more performant, but are more opaque, harder to command externally, and less easy to integrate into larger systems.