FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metricsfor Automatic Text Generation

Understand

Fast and reliable evaluation metrics are key to R&D progress.

• While traditional natural language generation metrics are fast, they are not very reliable.
• Conversely, new metrics based on large pretrained language models are much more reliable, but require significant computational resources.
• In this paper, we propose FrugalScore, an approach to learn a fixed, low cost version of any expensive NLG metric, while retaining most of its original performance.