Human-AI Collaborative Essay Scoring: A Dual-Process Framework with LLMs

Understand

Receiving timely and personalized feedback is essential for second-language learners, especially when human instructors are unavailable.

• This study explores the effectiveness of Large Language Models (LLMs), including both proprietary and open-source models, for Automated Essay Scoring (AES).
• Through extensive experiments with public and private datasets, we find that while LLMs do not surpass conventional state-of-the-art (SOTA) grading models in performance, they exhibit notable consistency, generalizability, and explainability.
• We propose an open-source LLM-based AES system, inspired by the dual-process theory.