IQ-Learn: Inverse soft-Q Learning for Imitation

Understand

In many sequential decision-making problems (e.g., robotics control, game playing, sequential prediction), human or expert data is available containing useful information about the task.

• However, imitation learning (IL) from a small amount of expert data can be challenging in high-dimensional environments with complex dynamics.
• Behavioral cloning is a simple method that is widely used due to its simplicity of implementation and stable convergence but doesn't utilize any information involving the environment's dynamics.
• Many existing methods that exploit dynamics information are difficult to train in practice due to an adversarial optimization process over reward and policy approximators or biased, high variance gradient estimators.