Planning with Sequence Models through Iterative Energy Minimization

Understand

Recent works have shown that sequence modeling can be effectively used to train reinforcement learning (RL) policies.

• However, the success of applying existing sequence models to planning, in which we wish to obtain a trajectory of actions to reach some goal, is less straightforward.
• The typical autoregressive generation procedures of sequence models preclude sequential refinement of earlier steps, which limits the effectiveness of a predicted plan.
• In this paper, we suggest an approach towards integrating planning with sequence models based on the idea of iterative energy minimization, and illustrate how such a procedure leads to improved RL performance across different tasks.