Compare without Despair: Reliable Preference Evaluation with Generation Separability

Bleu: a method for automatic evaluation of machine translation

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002

Earlier work this paper cites.

ROUGE: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

Abstractive text summarization using sequence-to-sequence RNNs and beyond

Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Çağlar Gulçehre, and Bing Xiang. 2016 · 2016

Earlier work this paper cites.

Agreement is overrated: A plea for correlation to assess human evaluation reliability

Jacopo Amidei, Paul Piwek, and Alistair Willis. 2019 · 2019

Earlier work this paper cites.

Findings of the 2019 conference on machine translation (WMT19)

Loïc Barrault, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Müller, Santanu Pal, Matt Post, and Marcos Zampieri. 2019 · 2019

Earlier work this paper cites.

Abductive commonsense reasoning

Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen-tau Yih, and Yejin Choi. 2019 · 2019

Earlier work this paper cites.

SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization

Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019 · 2019

Earlier work this paper cites.

How to compare summarizers without target length? pitfalls, solutions and re-examination of the neural summarization literature

Simeng Sun, Ori Shapira, Ido Dagan, and Ani Nenkova. 2019 · 2019

Earlier work this paper cites.

Bertscore: Evaluating text generation with bert

Original

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019 · 2019

Earlier work this paper cites.

Adversarial filters of dataset biases

Ronan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew Peters, Ashish Sabharwal, and Yejin Choi. 2020 · 2020

Earlier work this paper cites.

BiSECT: Learning to split and rephrase sentences with bitexts

Joongwon Kim, Mounica Maddela, Reno Kriz, Wei Xu, and Chris Callison-Burch. 2021 · 2021

Earlier work this paper cites.

On releasing annotator-level labels and information in datasets

Vinodkumar Prabhakaran, Aida Mostafazadeh Davani, and Mark Diaz. 2021 · 2021

Earlier work this paper cites.

Evaluation examples are not equally informative: How should that change NLP leaderboards?

Pedro Rodriguez, Joe Barrow, Alexander Miserlis Hoyle, John P. Lalor, Robin Jia, and Jordan Boyd-Graber. 2021 · 2021

Earlier work this paper cites.

Comparing test sets with item response theory

Clara Vania, Phu Mon Htut, William Huang, Dhara Mungra, Richard Yuanzhe Pang, Jason Phang, Haokun Liu, Kyunghyun Cho, and Samuel R. Bowman. 2021 · 2021

Cited alongside, same era.

The authenticity gap in human evaluation

Kawin Ethayarajh and Dan Jurafsky. 2022 · 2022

Cited alongside, same era.

Repairing the cracked foundation: A survey of obstacles in evaluation practices for generated text

Original

Sebastian Gehrmann, Elizabeth Clark, and Thibault Sellam. 2022 · 2022

Cited alongside, same era.

News summarization and evaluation in the era of gpt-3

Original

Tanya Goyal, Junyi Jessy Li, and Greg Durrett. 2022 · 2022

Cited alongside, same era.

On the limitations of dataset balancing: The lost battle against spurious correlations

Roy Schwartz and Gabriel Stanovsky. 2022 · 2022

Cited alongside, same era.

Large language models are not fair evaluators

Original

Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023 · 2023

Later among the works it cites.

Style over substance: Evaluation biases for large language models

Original

Minghao Wu and Alham Fikri Aji. 2023 · 2023

Later among the works it cites.

Evaluating large language models at evaluating instruction following

Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. 2023 · 2023

Later among the works it cites.

Benchmarking large language models for news summarization

Original

Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, and Tatsunori B Hashimoto. 2023 · 2023

Later among the works it cites.

Label-efficient model selection for text generation