Measuring Political Bias in Large Language Models: What Is Said and How It Is Said

Understand

We propose to measure political bias in LLMs by analyzing both the content and style of their generated content regarding political issues.

• Existing benchmarks and measures focus on gender and racial biases.
• However, political bias exists in LLMs and can lead to polarization and other harms in downstream applications.
• In order to provide transparency to users, we advocate that there should be fine-grained and explainable measures of political biases generated by LLMs.