Fetching the paper…

Reading the bibliography…

2025

Enhancing LLM Reliability via Explicit Knowledge Boundary Modeling

Zheng, Hang, Xu, Hongshen, Liu, Yuncong et al.

Original
PDF
Post on X

Understand

Large language models (LLMs) are prone to hallucination stemming from misaligned self-awareness, particularly when processing queries exceeding their knowledge boundaries.

While existing mitigation strategies employ uncertainty estimation or query rejection mechanisms, they suffer from computational efficiency and sacrificed helpfulness.
To address these issues, we propose the Explicit Knowledge Boundary Modeling (EKBM) framework, integrating fast and slow reasoning systems to harmonize reliability and usability.
The framework first employs a fast-thinking model to generate confidence-labeled responses, enabling immediate utilization of high-confidence outputs, whereas uncertain predictions trigger a slow refinement model for accuracy improvement.

Enhancing LLM Reliability via Explicit Knowledge Boundary Modeling · Around

Built on

Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset
Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan · 2020
Earlier work this paper cites.
Training verifiers to solve math word problems, 2021
Original
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman · 2021
Earlier work this paper cites.
Bitod: A bilingual multi-domain dataset for task-oriented dialogue modeling. arxiv 2021
Original
Z Lin, A Madotto, GI Winata, P Xu, F Jiang, Y Hu, C Shi, and P Fung · 2021
Earlier work this paper cites.
Multiwoz 2.4: A multi-domain task-oriented dialogue dataset with essential annotation corrections to improve state tracking evaluation
Original
Fanghua Ye, Jarana Manotumruksa, and Emine Yilmaz · 2021
Earlier work this paper cites.
Chatgpt: Fundamentals, applications and social impacts
Malak Abdullah, Alia Madain, and Yaser Jararweh · 2022
Earlier work this paper cites.
Language models (mostly) know what they know
Original
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al · 2022
Earlier work this paper cites.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al · 2022
Earlier work this paper cites.
A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions
Original
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al · 2023
Earlier work this paper cites.
Towards mitigating LLM hallucination via self reflection
Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung · 2023
Earlier work this paper cites.
Towards mitigating llm hallucination via self reflection
Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung · 2023
Earlier work this paper cites.
We’re afraid language models aren’t modeling ambiguity
Original
Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah A Smith, and Yejin Choi · 2023
Earlier work this paper cites.

Similar

Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models
Original
Potsawee Manakul, Adian Liusie, and Mark JF Gales · 2023
Cited alongside, same era.
Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback
Original
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning · 2023
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models, 2023
Original
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou · 2023
Cited alongside, same era.
Deepspeed-chat: Easy, fast and affordable rlhf training of chatgpt-like models at all scales
Original
Zhewei Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari, Xiaoxia Wu, Ammar Ahmad Awan, Jeff Rasley, Minjia Zhang, Conglong Li, Connor Holmes, et al · 2023
Cited alongside, same era.

Then

When to trust llms: Aligning confidence with response quality
Original
Shuchang Tao, Liuyi Yao, Hanxing Ding, Yuexiang Xie, Qi Cao, Fei Sun, Jinyang Gao, Huawei Shen, and Bolin Ding · 2024
Later among the works it cites.
A comprehensive survey of hallucination mitigation techniques in large language models
Original
SM Tonmoy, SM Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, and Amitava Das · 2024
Later among the works it cites.
Ualign: Leveraging uncertainty estimations for factuality alignment on large language models
Original
Boyang Xue, Fei Mi, Qi Zhu, Hongru Wang, Rui Wang, Sheng Wang, Erxin Yu, Xuming Hu, and Kam-Fai Wong · 2024
Later among the works it cites.
Qwen2. 5 technical report
Original
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al · 2024
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Siren’s song in the ai ocean: a survey on hallucination in large language models

Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al · 2023

Cited alongside, same era.

Quantifying uncertainty in answers from any language model and enhancing their trustworthiness

Jiuhai Chen and Jonas Mueller · 2024

Cited alongside, same era.

Teaching large language models to express knowledge boundary from their own signals

Lida Chen, Zujie Liang, Xintao Wang, Jiaqing Liang, Yanghua Xiao, Feng Wei, Jinglei Chen, Zhenghong Hao, Bing Han, and Wei Wang · 2024

Cited alongside, same era.

The llama 3 herd of models

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al · 2024

Cited alongside, same era.

A survey of uncertainty estimation in llms: Theory meets practice

Hsiu-Yuan Huang, Yutong Yang, Zhaoxi Zhang, Sanwoo Lee, and Yunfang Wu · 2024

Cited alongside, same era.

Gpt-4o system card

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al · 2024

Cited alongside, same era.

Inference-time intervention: Eliciting truthful answers from a language model

Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg

Cited in the paper.

Knowledge boundary of large language models: A survey

Moxin Li, Yong Zhao, Yang Deng, Wenxuan Zhang, Shuaiyi Li, Wenya Xie, See-Kiong Ng, and Tat-Seng Chua

Cited in the paper.

Benchmarking knowledge boundary for large language model: A different perspective on model evaluation

Xunjian Yin, Xu Zhang, Jie Ruan, and Xiaojun Wan · 2024

Later among the works it cites.

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

Daya Guo DeepSeek-AI, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al · 2025

Closest in time.

Discovering clone negatives via adaptive contrastive learning for image-text matching

Renjie Pan, Jihao Dong, and Hua Yang · 2025

Closest in time.

An automatic graph construction framework based on large language models for recommendation, 2025

Rong Shan, Jianghao Lin, Chenxu Zhu, Bo Chen, Menghui Zhu, Kangning Zhang, Jieming Zhu, Ruiming Tang, Yong Yu, and Weinan Zhang · 2025

Closest in time.

Reducing tool hallucination via reliability alignment, 2025

Hongshen Xu, Zichen Zhu, Lei Pan, Zihan Wang, Su Zhu, Da Ma, Ruisheng Cao, Lu Chen, and Kai Yu · 2025

Closest in time.

Relying on the unreliable: The impact of language models’ reluctance to express uncertainty

Kaitlyn Zhou, Jena D Hwang, Xiang Ren, and Maarten Sap · 2048

Closest in time.