A Group Fairness Lens for Large Language Models

Understand

The need to assess LLMs for bias and fairness is critical, with current evaluations often being narrow, missing a broad categorical view.

• In this paper, we propose evaluating the bias and fairness of LLMs from a group fairness lens using a novel hierarchical schema characterizing diverse social groups.
• Specifically, we construct a dataset, GFAIR, encapsulating target-attribute combinations across multiple dimensions.
• Moreover, we introduce statement organization, a new open-ended text generation task, to uncover complex biases in LLMs.