Who's Your Judge? On the Detectability of LLM-Generated Judgments

Understand

Large Language Model (LLM)-based judgments leverage powerful LLMs to efficiently evaluate candidate content and provide judgment scores.

• However, the inherent biases and vulnerabilities of LLM-generated judgments raise concerns, underscoring the urgent need for distinguishing them in sensitive scenarios like academic peer reviewing.
• In this work, we propose and formalize the task of judgment detection and systematically investigate the detectability of LLM-generated judgments.
• Unlike LLM-generated text detection, judgment detection relies solely on judgment scores and candidates, reflecting real-world scenarios where textual feedback is often unavailable in the detection process.