What does ChatGPT return about human values? Exploring value bias in ChatGPT using a descriptive value theory

Understand

There has been concern about ideological basis and possible discrimination in text generated by Large Language Models (LLMs).

• We test possible value biases in ChatGPT using a psychological value theory.
• We designed a simple experiment in which we used a number of different probes derived from the Schwartz basic value theory (items from the revised Portrait Value Questionnaire, the value type definitions, value names).
• We prompted ChatGPT via the OpenAI API repeatedly to generate text and then analyzed the generated corpus for value content with a theory-driven value dictionary using a bag of words approach.