Fetching the paper…

Reading the bibliography…

2025

SycEval: Evaluating LLM Sycophancy

Fanous, Aaron, Goldberg, Jacob, Agarwal, Ank A. et al.

Original
PDF
Post on X

Understand

Large language models (LLMs) are increasingly applied in educational, clinical, and professional settings, but their tendency for sycophancy -- prioritizing user agreement over independent reasoning -- poses risks to reliability.

This study introduces a framework to evaluate sycophantic behavior in ChatGPT-4o, Claude-Sonnet, and Gemini-1.5-Pro across AMPS (mathematics) and MedQuad (medical advice) datasets.
Sycophantic behavior was observed in 58.19% of cases, with Gemini exhibiting the highest rate (62.47%) and ChatGPT the lowest (56.71%).
Progressive sycophancy, leading to correct answers, occurred in 43.52% of cases, while regressive sycophancy, leading to incorrect answers, was observed in 14.66%.

SycEval: Evaluating LLM Sycophancy · Around

Built on

A Question-Entailment Approach to Question Answering
Ben Abacha, A.; and Demner-Fushman, D. 2019 · 2019
Earlier work this paper cites.
Measuring Mathematical Problem Solving With the MATH Dataset
Original
Hendrycks, D.; Burns, C.; Kadavath, S.; et al. 2021 · 2021
Earlier work this paper cites.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Original
Casper, S.; Davies, X.; Shi, C.; Gilbert, T. K.; Scheurer, J.; et al. 2023 · 2023
Earlier work this paper cites.
Towards Understanding Sycophancy in Language Models
Original
Sharma, M.; Tong, M.; et al. 2023 · 2023
Earlier work this paper cites.
Flattering to Deceive: The Impact of Sycophantic Behavior on User Trust in Large Language Models
Original
Carro, M. V. 2024 · 2024
Earlier work this paper cites.

Similar

From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning
Original
Chen, W.; Huang, Z.; et al. 2024 · 2024
Cited alongside, same era.
Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
Original
Denison, C.; MacDiarmid, M.; et al. 2024 · 2024
Cited alongside, same era.
TrustLLM: Trustworthiness in Large Language Models
Original
Huang, Y.; Sun, L.; Wang, H.; et al. 2024 · 2024
Cited alongside, same era.
Sycophancy in Large Language Models: Causes and Mitigations
Original
Malmqvist, L. 2024 · 2024
Cited alongside, same era.

Then

Fostering effective hybrid human-LLM reasoning and decision making
Passerini, A.; Gema, A.; Minervini, P.; Sayin, B.; and Tentori, K. 2024 · 2024
Later among the works it cites.
When Large Language Models Contradict Humans? Large Language Models’ Sycophantic Behaviour
Original
Ranaldi, L.; and Pucci, G. 2024 · 2024
Later among the works it cites.
Measuring Sycophancy of Language Models in Multi-turn Dialogues
Original
Hong, J.; Byun, G.; Kim, S.; and Shu, K. 2025 · 2025
Closest in time.
Ollama Python Client Library
Ollama. 2025 · 2025
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…