Revisiting Summarization Evaluation for Scientific Articles

Understand

Evaluation of text summarization approaches have been mostly based on metrics that measure similarities of system generated summaries with a set of human written gold-standard summaries.

• The most widely used metric in summarization evaluation has been the ROUGE family.
• ROUGE solely relies on lexical overlaps between the terms and phrases in the sentences; therefore, in cases of terminology variations and paraphrasing, ROUGE is not as effective.
• Scientific article summarization is one such case that is different from general domain summarization (e.g.