Can Large Language Models Capture Dissenting Human Voices?

Understand

Large language models (LLMs) have shown impressive achievements in solving a broad range of tasks.

• Augmented by instruction fine-tuning, LLMs have also been shown to generalize in zero-shot settings as well.
• However, whether LLMs closely align with the human disagreement distribution has not been well-studied, especially within the scope of natural language inference (NLI).
• In this paper, we evaluate the performance and alignment of LLM distribution with humans using two different techniques to estimate the multinomial distribution: Monte Carlo Estimation (MCE) and Log Probability Estimation (LPE).