White Men Lead, Black Women Help? Benchmarking and Mitigating Language Agency Social Biases in LLMs

Understand

Social biases can manifest in language agency.

• However, very limited research has investigated such biases in Large Language Model (LLM)-generated content.
• In addition, previous works often rely on string-matching techniques to identify agentic and communal words within texts, falling short of accurately classifying language agency.
• We introduce the Language Agency Bias Evaluation (LABE) benchmark, which comprehensively evaluates biases in LLMs by analyzing agency levels attributed to different demographic groups in model generations.