Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning

Understand

The performance of adversarial dialogue generation models relies on the quality of the reward signal produced by the discriminator.

• The reward signal from a poor discriminator can be very sparse and unstable, which may lead the generator to fall into a local optimum or to produce nonsense replies.
• To alleviate the first problem, we first extend a recently proposed adversarial dialogue generation method to an adversarial imitation learning solution.
• Then, in the framework of adversarial inverse reinforcement learning, we propose a new reward model for dialogue generation that can provide a more accurate and precise reward signal for generator training.