The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making

Understand

As large language models (LLMs) become increasingly integrated into society, their alignment with human morals is crucial.

• To better understand this alignment, we created a large corpus of human- and LLM-generated responses to various moral scenarios.
• We found a misalignment between human and LLM moral assessments; although both LLMs and humans tended to reject morally complex utilitarian dilemmas, LLMs were more sensitive to personal framing.
• We then conducted a quantitative user study involving 230 participants (N=230), who evaluated these responses by determining whether they were AI-generated and assessed their agreement with the responses.