Human2Robot: Learning Robot Actions from Paired Human-Robot Videos

Understand

Distilling knowledge from human demonstrations is a promising way for robots to learn and act.

• Existing methods, which often rely on coarsely-aligned video pairs, are typically constrained to learning global or task-level features.
• As a result, they tend to neglect the fine-grained frame-level dynamics required for complex manipulation and generalization to novel tasks.
• We posit that this limitation stems from a vicious circle of inadequate datasets and the methods they inspire.