Towards a Metric for Automated Conversational Dialogue System Evaluation and Improvement

Understand

We present "AutoJudge", an automated evaluation method for conversational dialogue systems.

• The method works by first generating dialogues based on self-talk, i.e.
• dialogue systems talking to itself.
• Then, it uses human ratings on these dialogues to train an automated judgement model.