World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic Manipulation

Understand

Robotic manipulation policies are commonly initialized through imitation learning, but their performance is limited by the scarcity and narrow coverage of expert data.

• Reinforcement learning can refine polices to alleviate this limitation, yet real-robot training is costly and unsafe, while training in simulators suffers from the sim-to-real gap.
• Recent advances in generative models have demonstrated remarkable capabilities in real-world simulation, with diffusion models in particular excelling at generation.
• This raises the question of how diffusion model-based world models can be combined to enhance pre-trained policies in robotic manipulation.