TRAIL: Near-Optimal Imitation Learning with Suboptimal Data

Understand

The aim in imitation learning is to learn effective policies by utilizing near-optimal expert demonstrations.

• However, high-quality demonstrations from human experts can be expensive to obtain in large numbers.
• On the other hand, it is often much easier to obtain large quantities of suboptimal or task-agnostic trajectories, which are not useful for direct imitation, but can nevertheless provide insight into the dynamical structure of the environment, showing what could be done in the environment even if not what should be done.
• We ask the question, is it possible to utilize such suboptimal offline datasets to facilitate provably improved downstream imitation learning? In this work, we answer this question affirmatively and present training objectives that use offline datasets to learn a factored transition model whose structure enables the extraction of a latent action space.