A Robotic Skill Learning System Built Upon Diffusion Policies and Foundation Models

Understand

In this paper, we build upon two major recent developments in the field, Diffusion Policies for visuomotor manipulation and large pre-trained multimodal foundational models to obtain a robotic skill learning system.

• The system can obtain new skills via the behavioral cloning approach of visuomotor diffusion policies given teleoperated demonstrations.
• Foundational models are being used to perform skill selection given the user's prompt in natural language.
• Before executing a skill the foundational model performs a precondition check given an observation of the workspace.