GATE: A Challenge Set for Gender-Ambiguous Translation Examples

Understand

Although recent years have brought significant progress in improving translation of unambiguously gendered sentences, translation of ambiguously gendered input remains relatively unexplored.

• When source gender is ambiguous, machine translation models typically default to stereotypical gender roles, perpetuating harmful bias.
• Recent work has led to the development of "gender rewriters" that generate alternative gender translations on such ambiguous inputs, but such systems are plagued by poor linguistic coverage.
• To encourage better performance on this task we present and release GATE, a linguistically diverse corpus of gender-ambiguous source sentences along with multiple alternative target language translations.