Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models

Understand

We use a dataset of U.S.

• first names with labels based on predominant gender and racial group to examine the effect of training corpus frequency on tokenization, contextualization, similarity to initial representation, and bias in BERT, GPT-2, T5, and XLNet.
• We show that predominantly female and non-white names are less frequent in the training corpora of these four language models.
• We find that infrequent names are more self-similar across contexts, with Spearman's r between frequency and self-similarity as low as -.763.