A Comparative Study of Quality Evaluation Methods for Text Summarization

Understand

Evaluating text summarization has been a challenging task in natural language processing (NLP).

• Automatic metrics which heavily rely on reference summaries are not suitable in many situations, while human evaluation is time-consuming and labor-intensive.
• To bridge this gap, this paper proposes a novel method based on large language models (LLMs) for evaluating text summarization.
• We also conducts a comparative study on eight automatic metrics, human evaluation, and our proposed LLM-based method.