Evaluating Gender Bias in Natural Language Inference

Understand

Gender-bias stereotypes have recently raised significant ethical concerns in natural language processing.

• However, progress in detection and evaluation of gender bias in natural language understanding through inference is limited and requires further investigation.
• In this work, we propose an evaluation methodology to measure these biases by constructing a challenge task that involves pairing gender-neutral premises against a gender-specific hypothesis.
• We use our challenge task to investigate state-of-the-art NLI models on the presence of gender stereotypes using occupations.