Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns

Understand

Coreference resolution is an important task for natural language understanding, and the resolution of ambiguous pronouns a longstanding challenge.

• Nonetheless, existing corpora do not capture ambiguous pronouns in sufficient volume or diversity to accurately indicate the practical utility of models.
• Furthermore, we find gender bias in existing corpora and systems favoring masculine entities.
• To address this, we present and release GAP, a gender-balanced labeled corpus of 8,908 ambiguous pronoun-name pairs sampled to provide diverse coverage of challenges posed by real-world text.