Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning

Understand

Offline Reinforcement Learning (RL) methods leverage previous experiences to learn better policies than the behavior policy used for data collection.

• However, they face challenges handling distribution shifts due to the lack of online interaction during training.
• To this end, we propose a novel method named State Reconstruction for Diffusion Policies (SRDP) that incorporates state reconstruction feature learning in the recent class of diffusion policies to address the problem of out-of-distribution (OOD) generalization.
• Our method promotes learning of generalizable state representation to alleviate the distribution shift caused by OOD states.