Fetching the paper…

Reading the bibliography…

2023

Chain-of-Verification Reduces Hallucination in Large Language Models

Dhuliawala, Shehzaad, Komeili, Mojtaba, Xu, Jing et al.

Original
PDF
Post on X

Understand

Generation of plausible yet incorrect factual information, termed hallucination, is an unsolved issue in large language models.

We study the ability of language models to deliberate on the responses they give in order to correct their mistakes.
We develop the Chain-of-Verification (CoVe) method whereby the model first (i) drafts an initial response; then (ii) plans verification questions to fact-check its draft; (iii) answers those questions independently so the answers are not biased by other responses; and (iv) generates its final verified response.
In experiments, we show CoVe decreases hallucinations across a variety of tasks, from list-based questions from Wikidata, closed book MultiSpanQA and longform text generation.

Chain-of-Verification Reduces Hallucination in Large Language Models · Around

Built on

The curious case of neural text degeneration
Original
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi · 2019
Earlier work this paper cites.
Language models as knowledge bases?
Original
Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019
Earlier work this paper cites.
On faithfulness and factuality in abstractive summarization
Original
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald · 2020
Earlier work this paper cites.
Recipes for building an open-domain chatbot
Original
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M Smith, et al · 2020
Earlier work this paper cites.
On exposure bias, hallucination and domain shift in neural machine translation
Original
Chaojun Wang and Rico Sennrich · 2020
Earlier work this paper cites.
Reason first, then respond: Modular generation for knowledge-infused dialogue
Original
Leonard Adolphs, Kurt Shuster, Jack Urbanek, Arthur Szlam, and Jason Weston · 2021
Earlier work this paper cites.
Retrieval augmentation reduces hallucination in conversation
Original
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston · 2021
Earlier work this paper cites.
Language models (mostly) know what they know
Original
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al · 2022
Earlier work this paper cites.
Multispanqa: A dataset for multi-span question answering
Haonan Li, Martin Tomko, Maria Vasardani, and Timothy Baldwin · 2022
Earlier work this paper cites.
Teaching language models to support answers with verified quotes
Original
Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, et al · 2022
Earlier work this paper cites.
Reducing conversational agents’ overconfidence through linguistic calibration
Sabrina J Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau · 2022
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Earlier work this paper cites.

Similar

Measuring and narrowing the compositionality gap in language models
Original
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis · 2022
Cited alongside, same era.
Self-consistency improves chain of thought reasoning in language models
Original
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al · 2022
Cited alongside, same era.
Large language models are reasoners with self-verification
Original
Yixuan Weng, Minjun Zhu, Shizhu He, Kang Liu, and Jun Zhao · 2022
Cited alongside, same era.
Do language models know when they’re hallucinating references?

Then

Quest: A retrieval dataset of entity-seeking queries with implicit set operations
Original
Chaitanya Malaviya, Peter Shaw, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2023
Closest in time.
Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models
Original
Potsawee Manakul, Adian Liusie, and Mark JF Gales · 2023
Closest in time.
Selfcheck: Using llms to zero-shot check their own step-by-step reasoning
Original
Ning Miao, Yee Whye Teh, and Tom Rainforth · 2023
Closest in time.
Factscore: Fine-grained atomic evaluation of factual precision in long form text generation
Original
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

Ayush Agrawal, Lester Mackey, and Adam Tauman Kalai · 2023

Cited alongside, same era.

Lm vs lm: Detecting factual errors via cross examination

Roi Cohen, May Hamri, Mor Geva, and Amir Globerson · 2023

Cited alongside, same era.

Truth-o-meter: Collaborating with llm in fighting its hallucinations

Boris A Galitsky · 2023

Cited alongside, same era.

Rarr: Researching and revising what language models say, using language models

Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al · 2023

Cited alongside, same era.

Survey of hallucination in natural language generation

Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung · 2023

Cited alongside, same era.

Learning to reason and memorize with self-notes

Jack Lanchantin, Shubham Toshniwal, Jason Weston, Arthur Szlam, and Sainbayar Sukhbaatar · 2023

Cited alongside, same era.

Inference-time intervention: Eliciting truthful answers from a language model

Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg · 2023

Cited alongside, same era.

Deductive verification of chain-of-thought reasoning

Original

Zhan Ling, Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, and Hao Su · 2023

Cited alongside, same era.

Check your facts and try again: Improving large language models with external knowledge and automated feedback

Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al · 2023

Closest in time.

Question decomposition improves the faithfulness of model-generated reasoning

Ansh Radhakrishnan, Karina Nguyen, Anna Chen, Carol Chen, Carson Denison, Danny Hernandez, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilė Lukošiūtė, et al · 2023

Closest in time.

Measuring attribution in natural language generation models

Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, and David Reitter · 2023

Closest in time.

Factually consistent summarization via reinforcement learning with textual entailment feedback

Paul Roit, Johan Ferret, Lior Shani, Roee Aharoni, Geoffrey Cideron, Robert Dadashi, Matthieu Geist, Sertan Girgin, Léonard Hussenot, Orgad Keller, et al · 2023

Closest in time.

A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation

Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu · 2023

Closest in time.

Fine-grained human feedback gives better rewards for language model training

Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A Smith, Mari Ostendorf, and Hannaneh Hajishirzi · 2023

Closest in time.

Improving language models via plug-and-play retrieval feedback

Wenhao Yu, Zhihan Zhang, Zhenwen Liang, Meng Jiang, and Ashish Sabharwal · 2023

Closest in time.

How language model hallucinations can snowball

Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A Smith · 2023

Closest in time.

Verify-and-edit: A knowledge-enhanced chain-of-thought framework

Original

Ruochen Zhao, Xingxuan Li, Shafiq Joty, Chengwei Qin, and Lidong Bing · 2023

Closest in time.