Crowdsourcing Lightweight Pyramids for Manual Summary Evaluation

Duc 2003: Documents, tasks, and measures

NIST. 2003 · 2003

Earlier work this paper cites.

Rouge: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

Evaluating content selection in summarization: The pyramid method

Ani Nenkova and Rebecca Passonneau. 2004 · 2004

Earlier work this paper cites.

Overview of duc 2006

Hoa Trang Dang. 2006 · 2006

Earlier work this paper cites.

Pyramid annotation guide: Duc 2006

Rebecca Passonneau. 2006 · 2006

Earlier work this paper cites.

Applying the pyramid method in the 2006 document understanding conference

Rebecca Passonneau, Kathleen McKeown, Sergey Sigelman, and Adam Goodkind. 2006 · 2006

Earlier work this paper cites.

Non-expert evaluation of summarization systems is risky

Dan Gillick and Yang Liu. 2010 · 2010

Earlier work this paper cites.

On aggregating labels from multiple crowd workers to infer relevance of documents

Mehdi Hosseini, Ingemar J Cox, Nataša Milić-Frayling, Gabriella Kazai, and Vishwa Vinay. 2012 · 2012

Earlier work this paper cites.

An assessment of the accuracy of automatic evaluation in summarization

Karolina Owczarzak, John M Conroy, Hoa Trang Dang, and Ani Nenkova. 2012 · 2012

Cited alongside, same era.

Learning whom to trust with mace

Dirk Hovy, Taylor Berg-Kirkpatrick, Ashish Vaswani, and Eduard Hovy. 2013 · 2013

Cited alongside, same era.

Document understanding conferences

NIST. 2014 · 2014

Cited alongside, same era.

Linguistically debatable or just plain wrong?

Barbara Plank, Dirk Hovy, and Anders Søgaard. 2014 · 2014

Cited alongside, same era.

Abstractive text summarization using sequence-to-sequence rnns and beyond

Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gulcehre, and Bing Xiang. 2016 · 2016

Cited alongside, same era.

Peak: Pyramid evaluation via automated knowledge extraction

Qian Yang, Rebecca J. Passonneau, and Gerard de Melo. 2016 · 2016

Cited alongside, same era.

The price of debiasing automatic metrics in natural language evalaution

Arun Chaganty, Stephen Mussmann, and Percy Liang. 2018 · 2018

Later among the works it cites.

Controllable abstractive summarization

Angela Fan, David Grangier, and Michael Auli. 2018 · 2018

Later among the works it cites.

April: Interactively learning to summarise by combining active preference learning and reinforcement learning

Yang Gao, Christian M. Meyer, and Iryna Gurevych. 2018 · 2018

Later among the works it cites.

Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies

Max Grusky, Mor Naaman, and Yoav Artzi. 2018 · 2018

Later among the works it cites.

Automatic pyramid evaluation exploiting edu-based extractive reference summaries

Tsutomu Hirao, Hidetaka Kamigaito, and Masaaki Nagata. 2018 · 2018

Later among the works it cites.

Text analysis conference

NIST. 2018 · 2018