Guiding Online Reinforcement Learning with Action-Free Offline Pretraining

Understand

Offline RL methods have been shown to reduce the need for environment interaction by training agents using offline collected episodes.

• However, these methods typically require action information to be logged during data collection, which can be difficult or even impossible in some practical cases.
• In this paper, we investigate the potential of using action-free offline datasets to improve online reinforcement learning, name this problem Reinforcement Learning with Action-Free Offline Pretraining (AFP-RL).
• We introduce Action-Free Guide (AF-Guide), a method that guides online training by extracting knowledge from action-free offline datasets.