Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement

Understand

The advancement of large language models (LLMs) has outpaced traditional evaluation methodologies.

• This progress presents novel challenges, such as measuring human-like psychological constructs, moving beyond static and task-specific benchmarks, and establishing human-centered evaluation.
• These challenges intersect with psychometrics, the science of quantifying the intangible aspects of human psychology, such as personality, values, and intelligence.
• This review paper introduces and synthesizes the emerging interdisciplinary field of LLM Psychometrics, which leverages psychometric instruments, theories, and principles to evaluate, understand, and enhance LLMs.