Re-evaluating ADEM: A Deeper Look at Scoring Dialogue Responses

Understand

Automatically evaluating the quality of dialogue responses for unstructured domains is a challenging problem.

• ADEM(Lowe et al.
• 2017) formulated the automatic evaluation of dialogue systems as a learning problem and showed that such a model was able to predict responses which correlate significantly with human judgements, both at utterance and system level.
• Their system was shown to have beaten word-overlap metrics such as BLEU with large margins.