METAL: Towards Multilingual Meta-Evaluation

Understand

With the rising human-like precision of Large Language Models (LLMs) in numerous tasks, their utilization in a variety of real-world applications is becoming more prevalent.

• Several studies have shown that LLMs excel on many standard NLP benchmarks.
• However, it is challenging to evaluate LLMs due to test dataset contamination and the limitations of traditional metrics.
• Since human evaluations are difficult to collect, there is a growing interest in the community to use LLMs themselves as reference-free evaluators for subjective metrics.