Multi-Dimensional Evaluation of Text Summarization with In-Context Learning

Bleu: A method for automatic evaluation of machine translation

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002

Earlier work this paper cites.

ROUGE: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

Summeval: Re-evaluating summarization evaluation

Original

Alexander R Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2020 · 2007

Earlier work this paper cites.

Evaluating coherence in dialogue systems using entailment

Nouha Dziri, Ehsan Kamalloo, Kory Mathewson, and Osmar Zaiane. 2019 · 2019

Earlier work this paper cites.

MoverScore: Text generation evaluating with contextualized embeddings and earth mover distance

Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M. Meyer, and Steffen Eger. 2019 · 2019

Earlier work this paper cites.

Language models are few-shot learners

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 2020

Earlier work this paper cites.

GRADE: Automatic graph-enhanced coherence metric for evaluating open-domain dialogue systems

Lishan Huang, Zheng Ye, Jinghui Qin, Liang Lin, and Xiaodan Liang. 2020 · 2020

Earlier work this paper cites.

Evaluating the factual consistency of abstractive text summarization

Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020 · 2020

Cited alongside, same era.

BLEURT: Learning robust metrics for text generation

Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020 · 2020

Cited alongside, same era.

Asking and answering questions to evaluate the factual consistency of summaries

Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020 · 2020

Cited alongside, same era.

Bertscore: Evaluating text generation with bert

Tianyi Zhang, Varsha Kishore, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020 · 2020

Cited alongside, same era.

Compression, transduction, and creation: A unified framework for evaluating natural language generation

Mingkai Deng, Bowen Tan, Zhengzhong Liu, Eric Xing, and Zhiting Hu. 2021 · 2021

Cited alongside, same era.

Annotating and modeling fine-grained factuality in summarization

ExplainaBoard: An explainable leaderboard for NLP

Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaichen Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, and Graham Neubig. 2021 · 2021

Later among the works it cites.

Prompt programming for large language models: Beyond the few-shot paradigm

Laria Reynolds and Kyle McDonell. 2021 · 2021

Later among the works it cites.

Bartscore: Evaluating generated text as text generation

Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021 · 2021

Later among the works it cites.

News summarization and evaluation in the era of gpt-3

Tanya Goyal, Junyi Jessy Li, and Greg Durrett. 2022 · 2022

Later among the works it cites.

Rethinking the role of demonstrations: What makes in-context learning work?

Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022 · 2022

Later among the works it cites.

Towards a unified multi-dimensional evaluator for text generation