Prompt-Tuning Decision Transformer with Preference Ranking

Understand

Prompt-tuning has emerged as a promising method for adapting pre-trained models to downstream tasks or aligning with human preferences.

• Prompt learning is widely used in NLP but has limited applicability to RL due to the complex physical meaning and environment-specific information contained within RL prompts.
• These factors require supervised learning to imitate the demonstrations and may result in a loss of meaning after learning.
• Additionally, directly extending prompt-tuning approaches to RL is challenging because RL prompts guide agent behavior based on environmental modeling and analysis, rather than filling in missing information, making it unlikely that adjustments to the prompt format for downstream tasks, as in NLP, can yield significant improvements.