Fetching the paper…

Reading the bibliography…

2024

Non-Linear Inference Time Intervention: Improving LLM Truthfulness

Hoscilowicz, Jakub, Wiacek, Adam, Chojnacki, Jan et al.

Original
PDF
Post on X

Understand

In this work, we explore LLM's internal representation space to identify attention heads that contain the most truthful and accurate information.

We further developed the Inference Time Intervention (ITI) framework, which lets bias LLM without the need for fine-tuning.
The improvement manifests in introducing a non-linear multi-token probing and multi-token intervention: Non-Linear ITI (NL-ITI), which significantly enhances performance on evaluation benchmarks.
NL-ITI is tested on diverse multiple-choice datasets, including TruthfulQA, on which we report over 16% relative MC1 (accuracy of model pointing to the correct answer) improvement with respect to the baseline ITI results.

Non-Linear Inference Time Intervention: Improving LLM Truthfulness · Around

Built on

P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try arc, the ai2 reasoning challenge,” arXiv:1803.05457v1 , 2018
Original
2018
Earlier work this paper cites.
T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal, “Can a suit of armor conduct electricity? a new dataset for open book question answering,” in EMNLP , 2018
2018
Earlier work this paper cites.
A. Gokaslan, V. Cohen, E. Pavlick, and S. Tellex, “Openwebtext corpus,” http://Skylion007.github.io/OpenWebTextCorpus , 2019
2019
Earlier work this paper cites.
N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. de Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” arXiv Preprint, arXiv:1902.00751 , 2019
Original
2019
Earlier work this paper cites.
T. Pimentel, J. Valvoda, R. H. Maudslay, R. Zmigrod, A. Williams, and R. Cotterell, “Information-theoretic probing for linguistic structure,” arXiv Preprint, arXiv:2004.03061 , 2020
Original
2020
Earlier work this paper cites.
J. C. White, T. Pimentel, N. Saphra, and R. Cotterell, “A non-linear structural probe,” arXiv Preprint, arXiv:2105.10185 , 2021
Original
2021
Earlier work this paper cites.
D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,” Proceedings of the International Conference on Learning Representations (ICLR) , 2021
2021
Earlier work this paper cites.
T. Hartvigsen, S. Gabriel, H. Palangi, M. Sap, D. Ray, and E. Kamar, “Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection,” 2022
2022
Earlier work this paper cites.
A. Parrish, A. Chen, N. Nangia, V. Padmakumar, J. Phang, J. Thompson, P. M. Htut, and S. Bowman, “BBQ: A hand-built bias benchmark for question answering,” in Findings of the Association for Computational Linguistics: ACL 2022 , S. Muresan, P. Nakov, and A. Villavicencio, Eds. Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 2086–2105. [Online]. Available: https://aclanthology.org/2022.findings-acl.165
2022
Earlier work this paper cites.

Similar

S. Lin, J. Hilton, and O. Evans, “Truthfulqa: Measuring how models mimic human falsehoods,” 2022
2022
Cited alongside, same era.
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions with human feedback,” arXiv Preprint, arXiv:2203.02155 , 2022
Original
2022
Cited alongside, same era.
S. Kadavath, T. Conerly, A. Askell, T. Henighan, D. Drain, E. Perez, N. Schiefer, Z. Hatfield-Dodds, N. DasSarma, E. Tran-Johnson, S. Johnston, S. El-Showk, A. Jones, N. Elhage, T. Hume, A. Chen, Y. Bai, S. Bowman, S. Fort, D. Ganguli, D. Hernandez, J. Jacobson, J. Kernion, S. Kravec, L. Lovitt, K. Ndousse, C. Olsson, S. Ringer, D. Amodei, T. Brown, J. Clark, N. Joseph, B. Mann, S. McCandlish, C. Olah, and J. Kaplan, “Language models (mostly) know what they know,” 2022
2022
Cited alongside, same era.
P. Manakul, A. Liusie, and M. J. F. Gales, “SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models,” arXiv Preprint, arXiv:2303.08896 , 2023

Then

H. Lee, S. Phatale, H. Mansoor, T. Mesnard, J. Ferret, K. Lu, C. Bishop, E. Hall, V. Carbune, A. Rastogi, and S. Prakash, “RLAIF: Scaling reinforcement learning from human feedback with AI feedback,” arXiv Preprint, arXiv:2309.00267 , 2023
Original
2023
Later among the works it cites.
S. Mao, N. Zhang, X. Wang, M. Wang, Y. Yao, Y. Jiang, P. Xie, F. Huang, and H. Chen, “Editing personality for LLMs,” arXiv Preprint, arXiv:2310.02168 , 2023
Original
2023
Later among the works it cites.
K. Pan and Y. Zeng, “Do LLMs possess a personality? Making the MBTI test an amazing evaluation for large language models,” arXiv Preprint, arXiv:2307.16180 , 2023
Original
2023
Later among the works it cites.
J. tse Huang, W. Wang, M. H. Lam, E. J. Li, W. Jiao, and M. R. Lyu, “Revisiting the reliability of psychological scales on large language models,” arXiv Preprint, arXiv:2305.19926 , 2023
Original
2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2023

Cited alongside, same era.

J. Li, X. Cheng, W. X. Zhao, J.-Y. Nie, and J.-R. Wen, “HaluEval: A large-scale hallucination evaluation benchmark for large language models,” arXiv Preprint, arXiv:2305.11747 , 2023

2023

Cited alongside, same era.

O. Shaikh, H. Zhang, W. Held, M. Bernstein, and D. Yang, “On second thought, let’s not think step by step! bias and toxicity in zero-shot reasoning,” arXiv Preprint, arXiv:2212.08061 , 2023

2023

Cited alongside, same era.

K.-C. Yeh, J.-A. Chi, D.-C. Lian, and S.-K. Hsieh, “Evaluating interfaced LLM bias,” in Proceedings of the 35th Conference on Computational Linguistics and Speech Processing (ROCLING 2023) , 2023, pp. 292–299

2023

Cited alongside, same era.

A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A.-K. Dombrowski, S. Goel, N. Li, M. J. Byun, Z. Wang, A. Mallen, S. Basart, S. Koyejo, D. Song, M. Fredrikson, J. Z. Kolter, and D. Hendrycks, “Representation engineering: A top-down approach to ai transparency,” 2023

2023

Cited alongside, same era.

K. Li, O. Patel, F. Viégas, H. Pfister, and M. Wattenberg, “Inference-time intervention: Eliciting truthful answers from a language model,” 2023

2023

Cited alongside, same era.

J. Hościłowicz, M. Sowański, P. Czubowski, and A. Janicki, “Can we use probing to better understand fine-tuning and knowledge distillation of the BERT nlu?” in Proceedings of the 15th International Conference on Agents and Artificial Intelligence (ICAART), Volume 3, Lisbon, Portugal, February 22-24, 2023 , A. P. Rocha, L. Steels, and H. J. van den Herik, Eds. SCITEPRESS, 2023, pp. 625–632. [Online]. Available: https://doi.org/10.5220/0011724900003393

2023

Later among the works it cites.

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language models,” 2023

2023

Later among the works it cites.

A. Taubenfeld, Y. Dover, R. Reichart, and A. Goldstein, “Systematic biases in LLM simulations of debates,” arXiv Preprint, arXiv:2402.04049 , 2024

2024

Closest in time.

Z. Chen, X. Sun, X. Jiao, F. Lian, Z. Kang, D. Wang, and C.-Z. Xu, “Truth forest: Toward multi-scale truthfulness in large language models through intervention without tuning,” arXiv Preprint, arXiv:2312.17484 , 2024

2024

Closest in time.