Contextual Transformer for Offline Meta Reinforcement Learning

Understand

The pretrain-finetuning paradigm in large-scale sequence models has made significant progress in natural language processing and computer vision tasks.

• However, such a paradigm is still hindered by several challenges in Reinforcement Learning (RL), including the lack of self-supervised pretraining algorithms based on offline data and efficient fine-tuning/prompt-tuning over unseen downstream tasks.
• In this work, we explore how prompts can improve sequence modeling-based offline reinforcement learning (offline-RL) algorithms.
• Firstly, we propose prompt tuning for offline RL, where a context vector sequence is concatenated with the input to guide the conditional policy generation.