NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark

Understand

In this position paper, we argue that the classical evaluation on Natural Language Processing (NLP) tasks using annotated benchmarks is in trouble.

• The worst kind of data contamination happens when a Large Language Model (LLM) is trained on the test split of a benchmark, and then evaluated in the same benchmark.
• The extent of the problem is unknown, as it is not straightforward to measure.
• Contamination causes an overestimation of the performance of a contaminated model in a target benchmark and associated task with respect to their non-contaminated counterparts.