DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching

Understand

In offline reinforcement learning (RL), the performance of the learned policy highly depends on the quality of offline datasets.

• However, in many cases, the offline dataset contains very limited optimal trajectories, which poses a challenge for offline RL algorithms as agents must acquire the ability to transit to high-reward regions.
• To address this issue, we introduce Diffusion-based Trajectory Stitching (DiffStitch), a novel diffusion-based data augmentation pipeline that systematically generates stitching transitions between trajectories.
• DiffStitch effectively connects low-reward trajectories with high-reward trajectories, forming globally optimal trajectories to address the challenges faced by offline RL algorithms.