ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Tasks

Understand

Evaluation of summarization tasks is extremely crucial to determining the quality of machine generated summaries.

• Over the last decade, ROUGE has become the standard automatic evaluation measure for evaluating summarization tasks.
• While ROUGE has been shown to be effective in capturing n-gram overlap between system and human composed summaries, there are several limitations with the existing ROUGE measures in terms of capturing synonymous concepts and coverage of topics.
• Thus, often times ROUGE scores do not reflect the true quality of summaries and prevents multi-faceted evaluation of summaries (i.e.