TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages

Understand

Confidently making progress on multilingual modeling requires challenging, trustworthy evaluations.

• We present TyDi QA---a question answering dataset covering 11 typologically diverse languages with 204K question-answer pairs.
• The languages of TyDi QA are diverse with regard to their typology---the set of linguistic features each language expresses---such that we expect models performing well on this set to generalize across a large number of the world's languages.
• We present a quantitative analysis of the data quality and example-level qualitative linguistic analyses of observed language phenomena that would not be found in English-only corpora.