Creativity Has Left the Chat: The Price of Debiasing Language Models

Understand

Large Language Models (LLMs) have revolutionized natural language processing but can exhibit biases and may generate toxic content.

• While alignment techniques like Reinforcement Learning from Human Feedback (RLHF) reduce these issues, their impact on creativity, defined as syntactic and semantic diversity, remains unexplored.
• We investigate the unintended consequences of RLHF on the creativity of LLMs through three experiments focusing on the Llama-2 series.
• Our findings reveal that aligned models exhibit lower entropy in token predictions, form distinct clusters in the embedding space, and gravitate towards "attractor states", indicating limited output diversity.