Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition

Understand

Existing research on fairness evaluation of document classification models mainly uses synthetic monolingual data without ground truth for author demographic attributes.

• In this work, we assemble and publish a multilingual Twitter corpus for the task of hate speech detection with inferred four author demographic factors: age, country, gender and race/ethnicity.
• The corpus covers five languages: English, Italian, Polish, Portuguese and Spanish.
• We evaluate the inferred demographic labels with a crowdsourcing platform, Figure Eight.