When Persuasion Overrides Truth in Multi-Agent LLM Debates: Introducing a Confidence-Weighted Persuasion Override Rate (CW-POR)

Built on

AI safety via debate
Original
G. Irving, P. F. Christiano, and D. Amodei · 2018
Earlier work this paper cites.
How can we know when language models know? On the calibration of language models for QA
Z. Jiang, P. Xia, D. Misra, Q. Lei, N. Shalom, R. Nallapati, et al · 2021
Earlier work this paper cites.
TruthfulQA: Measuring how models mimic human falsehoods
Original
S. Lin, J. Hilton, and O. Evans · 2021
Earlier work this paper cites.
Language models (mostly) know what they know
S. Kadavath, T. Conerly, A. Askell, et al · 2022
Earlier work this paper cites.

Debate helps supervise unreliable experts
Original
J. Michael, S. Mahdi, D. Rein, et al · 2023
Cited alongside, same era.
ChatEval: Towards better LLM-based evaluators through multi-agent debate
C. M. Chan, W. Z. Chen, Y. S. Su, and X. Ma · 2023
Cited alongside, same era.
GPT-4 technical report
OpenAI · 2023
Cited alongside, same era.
Survey of hallucination in natural language generation
Z. Ji, et al · 2023
Cited alongside, same era.

On scalable oversight with weak LLMs judging strong LLMs
Z. Kenton, N. Y. Siegel, J. Kramar, et al · 2024
Later among the works it cites.
Adversarial multi-agent evaluation of large language models through iterative debates
Original
C. Bandi and A. Harrasse · 2024
Later among the works it cites.
"The Earth is Flat because…": Investigating LLMs’ belief towards misinformation via persuasive conversation
W. L. Chiang, Z. Li, Z. Lin, Y. Sheng, et al · 2024
Later among the works it cites.
The persuasive power of large language models
S. M. Breum, D. V. Egdal, V. G. Mortensen, A. G. Moller, and L. M. Aiello · 2024
Later among the works it cites.