Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructions

Understand

LLMs-as-a-judge is a recently popularized method which replaces human judgements in task evaluation (Zheng et al.

• 2024) with automatic evaluation using LLMs.
• Due to widespread use of RLHF (Reinforcement Learning from Human Feedback), state-of-the-art LLMs like GPT4 and Llama3 are expected to have strong alignment with human preferences when prompted for a quality judgement, such as the coherence of a text.
• While this seems beneficial, it is not clear whether the assessments by an LLM-as-a-judge constitute only an evaluation based on the instructions in the prompts, or reflect its preference for high-quality data similar to its fine-tune data.