Coherent Dialogue with Attention-based Language Models

Understand

We model coherent conversation continuation via RNN-based dialogue models equipped with a dynamic attention mechanism.

• Our attention-RNN language model dynamically increases the scope of attention on the history as the conversation continues, as opposed to standard attention (or alignment) models with a fixed input scope in a sequence-to-sequence model.
• This allows each generated word to be associated with the most relevant words in its corresponding conversation history.
• We evaluate the model on two popular dialogue datasets, the open-domain MovieTriples dataset and the closed-domain Ubuntu Troubleshoot dataset, and achieve significant improvements over the state-of-the-art and baselines on several metrics, including complementary diversity-based metrics, human evaluation, and qualitative visualizations.