PONE: A Novel Automatic Evaluation Metric for Open-Domain Generative Dialogue Systems

Understand

Open-domain generative dialogue systems have attracted considerable attention over the past few years.

• Currently, how to automatically evaluate them, is still a big challenge problem.
• As far as we know, there are three kinds of automatic methods to evaluate the open-domain generative dialogue systems: (1) Word-overlap-based metrics; (2) Embedding-based metrics; (3) Learning-based metrics.
• Due to the lack of systematic comparison, it is not clear which kind of metrics are more effective.