DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languages

Understand

Language technologies should be judged on their usefulness in real-world use cases.

• An often overlooked aspect in natural language processing (NLP) research and evaluation is language variation in the form of non-standard dialects or language varieties (hereafter, varieties).
• Most NLP benchmarks are limited to standard language varieties.
• To fill this gap, we propose DIALECTBENCH, the first-ever large-scale benchmark for NLP on varieties, which aggregates an extensive set of task-varied variety datasets (10 text-level tasks covering 281 varieties).