Uncovering Latent Biases in Text: Method and Application to Peer Review

Understand

Quantifying systematic disparities in numerical quantities such as employment rates and wages between population subgroups provides compelling evidence for the existence of societal biases.

• However, biases in the text written for members of different subgroups (such as in recommendation letters for male and non-male candidates), though widely reported anecdotally, remain challenging to quantify.
• In this work, we introduce a novel framework to quantify bias in text caused by the visibility of subgroup membership indicators.
• We develop a nonparametric estimation and inference procedure to estimate this bias.