NIL: No-data Imitation Learning by Leveraging Pre-trained Video Diffusion Models

Understand

Acquiring physically plausible motor skills across diverse and unconventional morphologies-including humanoid robots, quadrupeds, and animals-is essential for advancing character simulation and robotics.

• Traditional methods, such as reinforcement learning (RL) are task- and body-specific, require extensive reward function engineering, and do not generalize well.
• Imitation learning offers an alternative but relies heavily on high-quality expert demonstrations, which are difficult to obtain for non-human morphologies.
• Video diffusion models, on the other hand, are capable of generating realistic videos of various morphologies, from humans to ants.