Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learning

Understand

Teaching robots to autonomously complete everyday tasks remains a challenge.

• Imitation Learning (IL) is a powerful approach that imbues robots with skills via demonstrations, but is limited by the labor-intensive process of collecting teleoperated robot data.
• Human videos offer a scalable alternative, but it remains difficult to directly train IL policies from them due to the lack of robot action labels.
• To address this, we propose to represent actions as short-horizon 2D trajectories on an image.