Learning by Semantic Similarity Makes Abstractive Summarization Better

Understand

By harnessing pre-trained language models, summarization models had rapid progress recently.

• However, the models are mainly assessed by automatic evaluation metrics such as ROUGE.
• Although ROUGE is known for having a positive correlation with human evaluation scores, it has been criticized for its vulnerability and the gap between actual qualities.
• In this paper, we compare the generated summaries from recent LM, BART, and the reference summaries from a benchmark dataset, CNN/DM, using a crowd-sourced human evaluation metric.