Can We Trust the Performance Evaluation of Uncertainty Estimation Methods in Text Summarization?

Fine-tune bert for extractive summarization

Original

Yang Liu. 2019 · 1903

Earlier work this paper cites.

Bertscore: Evaluating text generation with bert

Original

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019 · 1904

Earlier work this paper cites.

This email could save your life: Introducing the task of email subject line generation

Original

Rui Zhang and Joel Tetreault. 2019 · 1906

Earlier work this paper cites.

Text summarization with pretrained encoders

Original

Yang Liu and Mirella Lapata. 2019 · 1908

Earlier work this paper cites.

Sentence-bert: Sentence embeddings using siamese bert-networks

Original

Nils Reimers and Iryna Gurevych. 2019 · 1908

Earlier work this paper cites.

Improving back-translation with uncertainty-based confidence estimation

Original

Shuo Wang, Yang Liu, Chao Wang, Huanbo Luan, and Maosong Sun. 2019 · 1909

Earlier work this paper cites.

Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension

Original

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019 · 1910

Earlier work this paper cites.

Uncertainty estimation in autoregressive structured prediction

Original

Andrey Malinin and Mark Gales. 2020 · 2002

Earlier work this paper cites.

Bleu: a method for automatic evaluation of machine translation

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002

Earlier work this paper cites.

The usefulness of the audit report in investment and financing decisions

Antonio Duréndez Gómez-Guillamón. 2003 · 2003

Earlier work this paper cites.

Rouge: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

Spearman correlation coefficients, differences between

Leann Myers and Maria J Sirois. 2004 · 2004

Earlier work this paper cites.

A survey automatic text summarization

Oguzhan Tas and Farzad Kiyani. 2007 · 2007

Earlier work this paper cites.

Extractive summarization using supervised and semi-supervised learning

Kam-Fai Wong, Mingli Wu, and Wenjie Li. 2008 · 2008

Earlier work this paper cites.

A survey of text summarization extractive techniques

Vishal Gupta and Gurpreet Singh Lehal. 2010 · 2010

Earlier work this paper cites.

Pearson’s correlation coefficient

Philip Sedgwick. 2012 · 2012

Earlier work this paper cites.

Dropout as a bayesian approximation: Representing model uncertainty in deep learning

Yarin Gal and Zoubin Ghahramani. 2016 · 2016

Earlier work this paper cites.

Abstractive text summarization using sequence-to-sequence rnns and beyond

Original

Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et al. 2016 · 2016

Cited alongside, same era.

Diverse beam search: Decoding diverse solutions from neural sequence models

Original

Ashwin K Vijayakumar, Michael Cogswell, Ramprasath R Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. 2016 · 2016

Cited alongside, same era.

A simple unified framework for detecting out-of-distribution samples and adversarial attacks

Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018 · 2018

Cited alongside, same era.

Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization

Original

Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018 · 2018

Cited alongside, same era.

Unsupervised quality estimation for neural machine translation

Calibrating sequence likelihood improves conditional language generation

Yao Zhao, Mikhail Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, and Peter J Liu. 2022 · 2022

Later among the works it cites.

Analysis of each components of glass samples based on the spearman correlation coefficient model

Xin Zheng, Yusi Feng, and Hongkai Chen. 2022 · 2022

Later among the works it cites.

Towards a unified multi-dimensional evaluator for text generation

Original

Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. 2022 · 2022

Later among the works it cites.

Lm-polygraph: Uncertainty estimation for language models

Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun, Artem Vazhentsev, Sergey Petrakov, Kirill Fedyanin, Daniil Vasilev, Elizaveta Goncharova, Alexander Panchenko, Maxim Panov, et al. 2023 · 2023

Later among the works it cites.

Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation

Original