Importance of Search and Evaluation Strategies in Neural Dialogue Modeling

Understand

We investigate the impact of search strategies in neural dialogue modeling.

• We first compare two standard search algorithms, greedy and beam search, as well as our newly proposed iterative beam search which produces a more diverse set of candidate responses.
• We evaluate these strategies in realistic full conversations with humans and propose a model-based Bayesian calibration to address annotator bias.
• These conversations are analyzed using two automatic metrics: log-probabilities assigned by the model and utterance diversity.