Combating False Negatives in Adversarial Imitation Learning

Understand

In adversarial imitation learning, a discriminator is trained to differentiate agent episodes from expert demonstrations representing the desired behavior.

• However, as the trained policy learns to be more successful, the negative examples (the ones produced by the agent) become increasingly similar to expert ones.
• Despite the fact that the task is successfully accomplished in some of the agent's trajectories, the discriminator is trained to output low values for them.
• We hypothesize that this inconsistent training signal for the discriminator can impede its learning, and consequently leads to worse overall performance of the agent.