Understanding the Extent to which Summarization Evaluation Metrics Measure the Information Quality of Summaries

Bertscore: Evaluating text generation with BERT

Original

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2019 · 1904

Earlier work this paper cites.

ROUGE: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

Evaluating content selection in summarization: The pyramid method

Ani Nenkova and Rebecca J. Passonneau. 2004 · 2004

Earlier work this paper cites.

Automated summarization evaluation with basic elements

Eduard H. Hovy, Chin-Yew Lin, Liang Zhou, and Junichi Fukumoto. 2006 · 2006

Earlier work this paper cites.

Overview of the tac 2008 update summarization task

Hoa Trang Dang and Karolina Owczarzak. 2008 · 2008

Earlier work this paper cites.

Open information extraction from the web

Oren Etzioni, Michele Banko, Stephen Soderland, and Daniel S Weld. 2008 · 2008

Earlier work this paper cites.

Summarization system evaluation revisited: N-gram graphs

George Giannakopoulos, Vangelis Karkaletsis, George A. Vouros, and Panagiotis Stamatopoulos. 2008 · 2008

Earlier work this paper cites.

Summarization evaluation using transformed basic elements

Stephen Tratz and Eduard H. Hovy. 2008 · 2008

Earlier work this paper cites.

Overview of the tac 2009 summarization track

Hoa Trang Dang and Karolina Owczarzak. 2009 · 2009

Earlier work this paper cites.

Autosummeng and memog in evaluating guided summaries

George Giannakopoulos and Vangelis Karkaletsis. 2011 · 2011

Cited alongside, same era.

Overview of the TAC 2011 summarization track: Guided task and AESOP task

Karolina Owczarzak and Hoa Trang Dang. 2011 · 2011

Cited alongside, same era.

Summary evaluation: Together we stand npower-ed

George Giannakopoulos and Vangelis Karkaletsis. 2013 · 2013

Cited alongside, same era.

Automatically assessing machine summary content without a gold standard

Annie Louis and Ani Nenkova. 2013 · 2013

Cited alongside, same era.

Automated pyramid scoring of summaries using distributional semantics

Rebecca J. Passonneau, Emily Chen, Weiwei Guo, and Dolores Perin. 2013 · 2013

Cited alongside, same era.

Meteor universal: Language specific translation evaluation for any target language

Michael J. Denkowski and Alon Lavie. 2014 · 2014

Get to the point: Summarization with pointer-generator networks

Abigail See, Peter J. Liu, and Christopher D. Manning. 2017 · 2017

Later among the works it cites.

Automatic pyramid evaluation exploiting edu-based extractive reference summaries

Tsutomu Hirao, Hidetaka Kamigaito, and Masaaki Nagata. 2018 · 2018

Later among the works it cites.

Automated pyramid summarization evaluation

Yanjun Gao, Chen Sun, and Rebecca J. Passonneau. 2019 · 2019

Later among the works it cites.

Text summarization with pretrained encoders

Yang Liu and Mirella Lapata. 2019 · 2019

Later among the works it cites.

Crowdsourcing lightweight pyramids for manual summary evaluation

Ori Shapira, David Gabay, Yang Gao, Hadar Ronen, Ramakanth Pasunuru, Mohit Bansal, Yael Amsterdamer, and Ido Dagan. 2019 · 2019

Later among the works it cites.

Moverscore: Text generation evaluating with contextualized embeddings and earth mover distance