A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learning

Understand

Offline reinforcement learning (RL) provides a promising solution to learning an agent fully relying on a data-driven paradigm.

• However, constrained by the limited quality of the offline dataset, its performance is often sub-optimal.
• Therefore, it is desired to further finetune the agent via extra online interactions before deployment.
• Unfortunately, offline-to-online RL can be challenging due to two main challenges: constrained exploratory behavior and state-action distribution shift.