Probing LLM Hallucination from Within: Perturbation-Driven Approach via Internal Knowledge

The curious case of neural text degeneration

Original

Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019 · 1904

Earlier work this paper cites.

Large language models in medicine

Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023 · 1940

Earlier work this paper cites.

The Kolmogorov-Smirnov test for goodness of fit

Frank J Massey Jr. 1951 · 1951

Earlier work this paper cites.

Scheduled sampling for sequence prediction with recurrent neural networks

Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015 · 2015

Earlier work this paper cites.

spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing

Matthew Honnibal and Ines Montani. 2017 · 2017

Earlier work this paper cites.

Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension

Original

Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017 · 2017

Earlier work this paper cites.

On NMT search errors and model errors: Cat got your tongue?

Felix Stahlberg and Bill Byrne. 2019 · 2019

Earlier work this paper cites.

Truthfulqa: Measuring how models mimic human falsehoods

Original

Stephanie Lin, Jacob Hilton, and Owain Evans. 2021 · 2021

Earlier work this paper cites.

Keywordmap: Attention-based visual exploration for keyword analysis

Yamei Tu, Jiayi Xu, and Han-Wei Shen. 2021 · 2021

Earlier work this paper cites.

The survey: Text generation models in deep learning

Touseef Iqbal and Shaima Qureshi. 2022 · 2022

Earlier work this paper cites.

Language models (mostly) know what they know

Original

Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022 · 2022

Earlier work this paper cites.

Locating and editing factual associations in gpt

Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022 · 2022

Earlier work this paper cites.

The internal state of an LLM knows when it’s lying

Original

Amos Azaria and Tom Mitchell. 2023 · 2023

Earlier work this paper cites.

Lawyer Used ChatGPT In Court—And Cited Fake Cases. A Judge Is Considering Sanctions

Molly Bohannon. 2023 · 2023

Earlier work this paper cites.

Hallucination detection: Robustly discerning reliable answers in large language models

Yuyan Chen, Qiang Fu, Yichen Yuan, Zhihao Wen, Ge Fan, Dayiheng Liu, Dongmei Zhang, Zhixu Li, and Yanghua Xiao. 2023 · 2023

Earlier work this paper cites.

Chainpoll: A high efficacy method for llm hallucination detection

Original

Robert Friel and Atindriyo Sanyal. 2023 · 2023

Earlier work this paper cites.

Sok: Memorization in general-purpose large language models

Original

Valentin Hartmann, Anshuman Suri, Vincent Bindschaedler, David Evans, Shruti Tople, and Robert West. 2023 · 2023

Earlier work this paper cites.

Mistral 7b

Original

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023 · 2023

Cited alongside, same era.

Large language models struggle to learn long-tail knowledge

Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023 · 2023

Cited alongside, same era.

Halueval: A large-scale hallucination evaluation benchmark for large language models

Original

Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023 · 2023

Cited alongside, same era.

When not to trust language models: Investigating effectiveness of parametric and non-parametric memories

Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023 · 2023

Cited alongside, same era.

Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models

A survey on llm-as-a-judge

Original

Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. 2024 · 2024

Closest in time.

A probabilistic framework for llm hallucination detection via belief tree propagation

Original

Bairu Hou, Yang Zhang, Jacob Andreas, and Shiyu Chang. 2024 · 2024

Closest in time.

Can knowledge editing really correct hallucinations?

Original

Baixiang Huang, Canyu Chen, Xiongxiao Xu, Ali Payani, and Kai Shu. 2024 · 2024

Closest in time.

LLM internal states reveal hallucination risk faced with a query

Ziwei Ji, Delong Chen, Etsuko Ishii, Samuel Cahyawijaya, Yejin Bang, Bryan Wilie, and Pascale Fung. 2024 · 2024

Closest in time.

LLM challenges and solutions

Uday Kamath, Kevin Keenan, Garrett Somers, and Sarah Sorenson. 2024 · 2024