The Surprising Effectiveness of Representation Learning for Visual Imitation

Understand

While visual imitation learning offers one of the most effective ways of learning from visual demonstrations, generalizing from them requires either hundreds of diverse demonstrations, task specific priors, or large, hard-to-train parametric models.

• One reason such complexities arise is because standard visual imitation frameworks try to solve two coupled problems at once: learning a succinct but good representation from the diverse visual data, while simultaneously learning to associate the demonstrated actions with such representations.
• Such joint learning causes an interdependence between these two problems, which often results in needing large amounts of demonstrations for learning.
• To address this challenge, we instead propose to decouple representation learning from behavior learning for visual imitation.