Probing the Robustness of Trained Metrics for Conversational Dialogue Systems

Policy gradient methods for reinforcement learning with function approximation

Richard S. Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999 · 1999

Earlier work this paper cites.

BLEU: a Method for Automatic Evaluation of Machine Translation

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002

Earlier work this paper cites.

Recipes for building an open-domain chatbot

Original

Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M Smith, et al. 2020 · 2004

Earlier work this paper cites.

Learning an unreferenced metric for online dialogue evaluation

Original

Koustuv Sinha, Prasanna Parthasarathi, Jasmine Wang, Ryan Lowe, William L Hamilton, and Joelle Pineau. 2020 · 2005

Earlier work this paper cites.

How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation

Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016 · 2016

Earlier work this paper cites.

DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset

Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017 · 2017

Earlier work this paper cites.

Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses

Ryan Lowe, Michael Noseworthy, Iulian Vlad Serban, Nicolas Angelard-Gontier, Yoshua Bengio, and Joelle Pineau. 2017 · 2017

Earlier work this paper cites.

Parlai: A dialog research software platform

Original

A. H. Miller, W. Feng, A. Fisch, J. Lu, D. Batra, A. Bordes, D. Parikh, and J. Weston. 2017 · 2017

Cited alongside, same era.

Why we need new evaluation metrics for NLG

Jekaterina Novikova, Ondřej Dušek, Amanda Cercas Curry, and Verena Rieser. 2017 · 2017

Cited alongside, same era.

Language Models are Unsupervised Multitask Learners

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2018 · 2018

Cited alongside, same era.

RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems

Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan. 2018 · 2018

Cited alongside, same era.

Personalizing Dialogue Agents: I have a dog, do you have pets too?

Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018 · 2018

Cited alongside, same era.

Re-Evaluating ADEM: A Deeper Look at Scoring Dialogue Responses

Ananya B Sai, Mithun Das Gupta, Mitesh M Khapra, and Mukundhan Srinivasan. 2019 · 2019

Later among the works it cites.

Survey on Evaluation Methods for Dialogue Systems

Jan Deriu, Alvaro Rodrigo, Arantxa Otegi, Guillermo Echegoyen, Sophie Rosset, Eneko Agirre, and Mark Cieliebak. 2020 · 2020

Later among the works it cites.

The second conversational intelligence challenge (convai2)

Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek, Douwe Kiela, Arthur Szlam, Iulian Serban, Ryan Lowe, Shrimai Prabhumoye, Alan W. Black, Alexander Rudnicky, Jason Williams, Joelle Pineau, Mikhail Burtsev, and Jason Weston. 2020 · 2020

Later among the works it cites.

USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generation

Shikib Mehri and Maxine Eskenazi. 2020 · 2020

Later among the works it cites.

An adversarially-learned turing test for dialog generation models

Original

Xiang Gao, Yizhe Zhang, Michel Galley, and Bill Dolan. 2021 · 2021

Later among the works it cites.