Fetching the paper…
Reading the bibliography…
The remarkable performance of large language models (LLMs) in content generation, coding, and common-sense reasoning has spurred widespread integration into many facets of society.
Variational Inference to Measure Model Uncertainty in Deep Neural Networks
Konstantin Posch, Jan Steinbrener, and Jürgen Pilz. 2019 · 1902
Earlier work this paper cites.
Monte Carlo sampling methods using Markov chains and their applications
W Keith Hastings. 1970 · 1970
Earlier work this paper cites.
Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods
John Platt et al · 1999
Earlier work this paper cites.
A natural logic inference system. In Proceedings of the 2nd workshop on inference in computational semantics (ICoS-2)
Yaroslav Fyodorov, Yoad Winter, and Nissim Francez. 2000 · 2000
Earlier work this paper cites.
Light-weight entailment checking for computational semantics. In Proc. of the third workshop on inference in computational semantics (ICoS-3)
Christof Monz and Maarten de Rijke. 2001 · 2001
Earlier work this paper cites.
Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Icml , Vol. 1. 609–616
Bianca Zadrozny and Charles Elkan. 2001 · 2001
Earlier work this paper cites.
Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining . 694–699
Bianca Zadrozny and Charles Elkan. 2002 · 2002
Earlier work this paper cites.
Entailment, intensionality and text understanding. In Proceedings of the HLT-NAACL 2003 workshop on Text meaning . 38–45
Cleo Condoravdi, Dick Crouch, Valeria De Paiva, Reinhard Stolle, and Daniel Bobrow. 2003 · 2003
Earlier work this paper cites.
Rouge: A package for automatic evaluation of summaries. In Text summarization branches out . 74–81
Chin-Yew Lin. 2004 · 2004
Earlier work this paper cites.
The pascal recognising textual entailment challenge. In Machine learning challenges workshop . Springer, 177–190
Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005 · 2005
Earlier work this paper cites.
Predicting good probabilities with supervised learning. In Proceedings of the 22nd international conference on Machine learning . 625–632
Alexandru Niculescu-Mizil and Rich Caruana. 2005 · 2005
Earlier work this paper cites.
Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining . 535–541
Cristian Buciluǎ, Rich Caruana, and Alexandru Niculescu-Mizil. 2006 · 2006
Earlier work this paper cites.
Elements of information theory
MTCAJ Thomas and A Thomas Joy. 2006 · 2006
Earlier work this paper cites.
Probabilistic forecasts, calibration and sharpness
Tilmann Gneiting, Fadoua Balabdaoui, and Adrian E Raftery. 2007 · 2007
Earlier work this paper cites.
Strictly proper scoring rules, prediction, and estimation
Tilmann Gneiting and Adrian E Raftery. 2007 · 2007
Earlier work this paper cites.
Modeling semantic containment and exclusion in natural language inference. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008) . 521–528
Bill MacCartney and Christopher D Manning. 2008 · 2008
Earlier work this paper cites.
An analysis of ensemble pruning techniques based on ordered aggregation
Gonzalo Martinez-Munoz, Daniel Hernández-Lobato, and Alberto Suárez. 2008 · 2008
Earlier work this paper cites.
A tutorial on conformal prediction
Glenn Shafer and Vladimir Vovk. 2008 · 2008
Earlier work this paper cites.
Align, disambiguate and walk: A unified approach for measuring semantic similarity. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 1341–1351
Mohammad Taher Pilehvar, David Jurgens, and Roberto Navigli. 2013 · 2013
Earlier work this paper cites.
Distilling the Knowledge in a Neural Network
Geoffrey Hinton. 2015 · 2015
Earlier work this paper cites.
Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence , Vol. 29
Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. 2015 · 2015
Earlier work this paper cites.
Combining diversity measures for ensemble pruning
George DC Cavalcanti, Luiz S Oliveira, Thiago JM Moura, and Guilherme V Carvalho. 2016 · 2016
Earlier work this paper cites.
Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning . PMLR, 1050–1059
Yarin Gal and Zoubin Ghahramani. 2016 · 2016
Earlier work this paper cites.
Generating text from structured data with application to the biography domain
Rémi Lebret, David Grangier, and Michael Auli. 2016 · 2016
Earlier work this paper cites.
A comparison of rule-based and machine learning approaches for classifying patient portal messages
Robert M Cronin, Daniel Fabbri, Joshua C Denny, S Trent Rosenbloom, and Gretchen Purcell Jackson. 2017 · 2017
Earlier work this paper cites.
Concrete dropout
Yarin Gal, Jiri Hron, and Alex Kendall. 2017 · 2017
Earlier work this paper cites.
On calibration of modern neural networks. In International conference on machine learning . PMLR, 1321–1330
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017 · 2017
Earlier work this paper cites.
Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension
Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017 · 2017
Earlier work this paper cites.
Race: Large-scale reading comprehension dataset from examinations
Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017 · 2017
Earlier work this paper cites.
Simple and scalable predictive uncertainty estimation using deep ensembles
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017 · 2017
Earlier work this paper cites.
Attention is all you need
A Vaswani. 2017 · 2017
Earlier work this paper cites.
A broad-coverage challenge corpus for sentence understanding through inference
Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017 · 2017
Earlier work this paper cites.
Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks. In Medical Imaging with Deep Learning
Murat Seckin Ayhan and Philipp Berens. 2018 · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin. 2018 · 2018
Earlier work this paper cites.
Margin & diversity based ordering ensemble pruning
Huaping Guo, Hongbing Liu, Ran Li, Changan Wu, Yibo Guo, and Mingliang Xu. 2018 · 2018
Earlier work this paper cites.
Hallucinations in neural machine translation
Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, and David Sussillo. 2018 · 2018
Earlier work this paper cites.
Improving Language Understanding by Generative Pre-Training
Alec Radford and Karthik Narasimhan. 2018 · 2018
Earlier work this paper cites.
FEVER: a large-scale dataset for fact extraction and VERification
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018 · 2018
Earlier work this paper cites.
Out-of-distribution detection using an ensemble of self supervised leave-out classifiers. In Proceedings of the European conference on computer vision (ECCV) . 550–564
Apoorv Vyas, Nataraj Jammalamadaka, Xia Zhu, Dipankar Das, Bharat Kaul, and Theodore L Willke. 2018 · 2018
Earlier work this paper cites.
HotpotQA: A dataset for diverse, explainable multi-hop question answering
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018 · 2018
Earlier work this paper cites.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu. 2019 · 2019
Earlier work this paper cites.
Measuring Calibration in Deep Learning.. In CVPR workshops , Vol. 2
Jeremy Nixon, Michael W Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. 2019 · 2019
Earlier work this paper cites.
Coqa: A conversational question answering challenge
Siva Reddy, Danqi Chen, and Christopher D Manning. 2019 · 2019
Earlier work this paper cites.
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
N Reimers. 2019 · 2019
Earlier work this paper cites.
Least ambiguous set-valued classifiers with bounded error levels
Mauricio Sadinle, Jing Lei, and Larry Wasserman. 2019 · 2019
Earlier work this paper cites.
Cxplain: Causal explanations for model interpretation under uncertainty
Patrick Schwab and Walter Karlen. 2019 · 2019
Earlier work this paper cites.
Hellaswag: Can a machine really finish your sentence?
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019 · 2019
Earlier work this paper cites.
Bertscore: Evaluating text generation with bert
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019 · 2019
Earlier work this paper cites.
Classification confidence estimation with test-time data-augmentation
Yuval Bahat and Gregory Shakhnarovich. 2020 · 2020
Earlier work this paper cites.
Language models are few-shot learners
Tom B Brown. 2020 · 2020
Earlier work this paper cites.
Calibration of pre-trained transformers
Shrey Desai and Greg Durrett. 2020 · 2020
Earlier work this paper cites.
Language-agnostic BERT sentence embedding
Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020 · 2020
Earlier work this paper cites.
Transformer feed-forward layers are key-value memories
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2020 · 2020
Earlier work this paper cites.
Deberta: Decoding-enhanced bert with disentangled attention
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020 · 2020
Earlier work this paper cites.
Measuring massive multitask language understanding
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020 · 2020
Earlier work this paper cites.
Calibrated language model fine-tuning for in-and out-of-distribution data
Lingkai Kong, Haoming Jiang, Yuchen Zhuang, Jie Lyu, Tuo Zhao, and Chao Zhang. 2020 · 2020
Earlier work this paper cites.
Gradients as a measure of uncertainty in neural networks. In 2020 IEEE International Conference on Image Processing (ICIP) . IEEE, 2416–2420
Jinsol Lee and Ghassan AlRegib. 2020 · 2020
Earlier work this paper cites.
A general framework for uncertainty estimation in deep learning
Antonio Loquercio, Mattia Segu, and Davide Scaramuzza. 2020 · 2020
Earlier work this paper cites.
Uncertainty estimation in autoregressive structured prediction
Andrey Malinin and Mark Gales. 2020 · 2020
Earlier work this paper cites.
BERT-based conformal predictor for sentiment analysis. In Conformal and Probabilistic Prediction and Applications . PMLR, 269–284
Lysimachos Maltoudoglou, Andreas Paisios, and Harris Papadopoulos. 2020 · 2020
Earlier work this paper cites.
AmbigQA: Answering ambiguous open-domain questions
Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020 · 2020
Earlier work this paper cites.
Document processing: Methods for semantic text similarity analysis. In 2020 international conference on INnovations in Intelligent SysTems and Applications (INISTA) . IEEE, 1–6
Abdul Wahab Qurashi, Violeta Holmes, and Anju P Johnson. 2020 · 2020
Earlier work this paper cites.
Controlling style in generated dialogue
Eric Michael Smith, Diana Gonzalez-Rico, Emily Dinan, and Y-Lan Boureau. 2020 · 2020
Earlier work this paper cites.
Building and evaluating open-domain dialogue corpora with clarifying questions
Mohammad Aliannejadi, Julia Kiseleva, Aleksandr Chuklin, Jeffrey Dalton, and Mikhail Burtsev. 2021 · 2021
Earlier work this paper cites.
Training verifiers to solve math word problems
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al · 2021
Earlier work this paper cites.
Measuring and improving consistency in pretrained language models
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021 · 2021
Earlier work this paper cites.
Did Aristotle use a laptop? a question answering benchmark with implicit reasoning strategies
Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021 · 2021
Earlier work this paper cites.
Measuring coding challenge competence with apps
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al · 2021
Earlier work this paper cites.
Lora: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021
Earlier work this paper cites.
On the importance of gradients for detecting distributional shifts in the wild
Rui Huang, Andrew Geng, and Yixuan Li. 2021 · 2021
Earlier work this paper cites.
Truthfulqa: Measuring how models mimic human falsehoods
Stephanie Lin, Jacob Hilton, and Owain Evans. 2021 · 2021
Earlier work this paper cites.
Uncertainty quantification and deep ensembles
Rahul Rahaman et al · 2021
Earlier work this paper cites.
Learning from the best: Rationalizing predictions by adversarial information calibration. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 35. 13771–13779
Lei Sha, Oana-Maria Camburu, and Thomas Lukasiewicz. 2021 · 2021
Earlier work this paper cites.
On hallucination and predictive uncertainty in conditional language generation
Yijun Xiao and William Yang Wang. 2021 · 2021
Earlier work this paper cites.
Bartscore: Evaluating generated text as text generation
Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021 · 2021
Earlier work this paper cites.
Zeyu Yun, Yubei Chen, Bruno A Olshausen, and Yann LeCun. 2021 · 2021
Earlier work this paper cites.
Calibrate before use: Improving few-shot performance of language models. In International conference on machine learning . PMLR, 12697–12706
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021 · 2021
Earlier work this paper cites.
Do as i can, not as i say: Grounding language in robotic affordances
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al · 2022
Earlier work this paper cites.
Probing classifiers: Promises, shortcomings, and advances
Yonatan Belinkov. 2022 · 2022
Earlier work this paper cites.
Rt-1: Robotics transformer for real-world control at scale
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al · 2022
Earlier work this paper cites.
Tomayto, tomahto. beyond token-level answer equivalence for question answering evaluation
Jannis Bulian, Christian Buck, Wojciech Gajewski, Benjamin Boerschinger, and Tal Schuster. 2022 · 2022
Earlier work this paper cites.
Discovering latent knowledge in language models without supervision
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2022 · 2022
Earlier work this paper cites.
A close look into the calibration of pre-trained language models
Yangyi Chen, Lifan Yuan, Ganqu Cui, Zhiyuan Liu, and Heng Ji. 2022 · 2022
Earlier work this paper cites.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al · 2022
Earlier work this paper cites.
How useful are gradients for ood detection really?
Conor Igoe, Youngseog Chung, Ian Char, and Jeff Schneider. 2022 · 2022
Earlier work this paper cites.
Hands-on Bayesian neural networks—A tutorial for deep learning users
Laurent Valentin Jospin, Hamid Laga, Farid Boussaid, Wray Buntine, and Mohammed Bennamoun. 2022 · 2022
Cited alongside, same era.
Language models (mostly) know what they know
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al · 2022
Cited alongside, same era.
Teaching models to express their uncertainty in words
Stephanie Lin, Jacob Hilton, and Owain Evans. 2022 · 2022
Cited alongside, same era.
FacTeR-Check: Semi-automated fact-checking through semantic similarity and natural language inference
Alejandro Martín, Javier Huertas-Tato, Álvaro Huertas-García, Guillermo Villar-Rodríguez, and David Camacho. 2022 · 2022
Cited alongside, same era.
Training-free uncertainty estimation for dense regression: Sensitivity as a surrogate. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 36. 10042–10050
Don’t Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration
Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran, and Yulia Tsvetkov. 2024 · 2024
Closest in time.
Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models
Javier Ferrando, Oscar Obeso, Senthooran Rajamanoharan, and Neel Nanda. 2024 · 2024
Closest in time.
Scaling and evaluating sparse autoencoders
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2024 · 2024
Closest in time.
A Survey of Confidence Estimation and Calibration in Large Language Models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) . 6577–6595
Jiahui Geng, Fengyu Cai, Yuxia Wang, Heinz Koeppl, Preslav Nakov, and Iryna Gurevych. 2024 · 2024
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Lu Mi, Hao Wang, Yonglong Tian, Hao He, and Nir N Shavit. 2022 · 2022
Cited alongside, same era.
Reducing conversational agents’ overconfidence through linguistic calibration
Sabrina J Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau. 2022 · 2022
Cited alongside, same era.
Fine-tuning language models via epistemic neural networks
Ian Osband, Seyed Mohammad Asghari, Benjamin Van Roy, Nat McAleese, John Aslanides, and Geoffrey Irving. 2022 · 2022
Cited alongside, same era.
Task ambiguity in humans and language models
Alex Tamkin, Kunal Handa, Avash Shrestha, and Noah Goodman. 2022 · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al · 2022
Cited alongside, same era.
Calibrating sequence likelihood improves conditional language generation. In The eleventh international conference on learning representations
Yao Zhao, Mikhail Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, and Peter J Liu. 2022 · 2022
Cited alongside, same era.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023
Cited alongside, same era.
Artificial hallucinations in ChatGPT: implications in scientific writing
Hussam Alkaissi and Samy I McFarlane. 2023 · 2023
Cited alongside, same era.
Tobias Groot and Matias Valdenegro-Toro. 2024 · 2024
Closest in time.
Towards uncertainty-aware language agent
Jiuzhou Han, Wray Buntine, and Ehsan Shareghi. 2024 · 2024
Closest in time.
Mitigating Hallucinations in LLM Using K-means Clustering of Synonym Semantic Relevance
Lin He and Keqin Li. 2024 · 2024
Closest in time.
Enhancing sequential recommendation via llm-based semantic embedding learning. In Companion Proceedings of the ACM on Web Conference 2024 . 103–111
Jun Hu, Wenwen Xia, Xiaolu Zhang, Chilin Fu, Weichang Wu, Zhaoxin Huan, Ang Li, Zuoli Tang, and Jun Zhou. 2024 · 2024
Closest in time.
A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice
Hsiu-Yuan Huang, Yutong Yang, Zhaoxi Zhang, Sanwoo Lee, and Yunfang Wu. 2024 · 2024
Closest in time.
Graph-based Uncertainty Metrics for Long-form Language Model Outputs
Mingjian Jiang, Yangjun Ruan, Prasanna Sattigeri, Salim Roukos, and Tatsunori Hashimoto. 2024 · 2024
Closest in time.
Experts Don’t Cheat: Learning What You Don’t Know By Predicting Pairs
Daniel D Johnson, Daniel Tarlow, David Duvenaud, and Chris J Maddison. 2024 · 2024
Closest in time.
Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreement
Jaehun Jung, Faeze Brahman, and Yejin Choi. 2024 · 2024
Closest in time.
Scope Ambiguities in Large Language Models
Gaurav Kamath, Sebastian Schuster, Sowmya Vajjala, and Siva Reddy. 2024 · 2024
Closest in time.
Large Language Models Must Be Taught to Know What They Don’t Know
Sanyam Kapoor, Nate Gruver, Manley Roberts, Katherine Collins, Arka Pal, Umang Bhatt, Adrian Weller, Samuel Dooley, Micah Goldblum, and Andrew Gordon Wilson. 2024 · 2024
Closest in time.
Gpt-4 passes the bar exam
Daniel Martin Katz, Michael James Bommarito, Shang Gao, and Pablo Arredondo. 2024 · 2024
Closest in time.
On the attribution of confidence to large language models
Geoff Keeling and Winnie Street. 2024 · 2024
Closest in time.
OpenVLA: An Open-Source Vision-Language-Action Model
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al · 2024
Closest in time.
" I’m Not Sure, But…": Examining the Impact of Large Language Models’ Uncertainty Expression on User Reliance and Trust. In The 2024 ACM Conference on Fairness, Accountability, and Transparency . 822–835
Sunnie SY Kim, Q Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, and Jennifer Wortman Vaughan. 2024a · 2024
Closest in time.
Aligning Uncertainty: Leveraging LLMs to Analyze Uncertainty Transfer in Text Summarization. In Proceedings of the 1st Workshop on Uncertainty-Aware NLP (UncertaiNLP 2024) . 41–61
Zahra Kolagar and Alessandra Zarcone. 2024 · 2024
Closest in time.
Semantic entropy probes: Robust and cheap hallucination detection in llms
Jannik Kossen, Jiatong Han, Muhammed Razzak, Lisa Schut, Shreshth Malik, and Yarin Gal. 2024 · 2024
Closest in time.
Can Large Language Models Achieve Calibration with In-Context Learning?. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models
Chengzu Li, Han Zhou, Goran Glavaš, Anna Korhonen, and Ivan Vulić. 2024b · 2024
Closest in time.
TRAQ: Trustworthy Retrieval Augmented Question Answering via Conformal Prediction. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) . 3799–3821
Shuo Li, Sangdon Park, Insup Lee, and Osbert Bastani. 2024a · 2024
Closest in time.
Introspective Planning: Guiding Language-Enabled Agents to Refine Their Own Uncertainty
Kaiqu Liang, Zixu Zhang, and Jaime Fernández Fisac. 2024 · 2024
Closest in time.
Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2
Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. 2024 · 2024
Closest in time.
Uncertainty decomposition and quantification for in-context learning of large language models
Chen Ling, Xujiang Zhao, Wei Cheng, Yanchi Liu, Yiyou Sun, Xuchao Zhang, Mika Oishi, Takao Osaki, Katsushi Matsuda, Jie Ji, et al · 2024
Closest in time.
On Calibration of LLM-based Guard Models for Reliable Content Moderation
Hongfu Liu, Hengguan Huang, Hao Wang, Xiangming Gu, and Ye Wang. 2024a · 2024
Closest in time.
Visual instruction tuning
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024c · 2024
Closest in time.
A survey on hallucination in large vision-language models
Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. 2024e · 2024
Closest in time.
Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach
Linyu Liu, Yu Pan, Xiaocheng Li, and Guanting Chen. 2024d · 2024
Closest in time.
Multi-group Uncertainty Quantification for Long-form Text Generation
Terrance Liu and Zhiwei Steven Wu. 2024 · 2024
Closest in time.
Calibrating large language models with sample consistency
Qing Lyu, Kumar Shridhar, Chaitanya Malaviya, Li Zhang, Yanai Elazar, Niket Tandon, Marianna Apidianaki, Mrinmaya Sachan, and Chris Callison-Burch. 2024 · 2024
Closest in time.
Factual Confidence of LLMs: on Reliability and Robustness of Current Estimators
Matéo Mahaut, Laura Aina, Paula Czarnowska, Momchil Hardalov, Thomas Müller, and Lluís Màrquez. 2024 · 2024
Closest in time.
Roco: Dialectic multi-robot collaboration with large language models. In 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 286–299
Zhao Mandi, Shreeya Jain, and Shuran Song. 2024 · 2024
Closest in time.
Don’t Forget Your Reward Values: Language Model Alignment via Value-based Calibration
Xin Mao, Feng-Lin Li, Huimin Xu, Wei Zhang, and Anh Tuan Luu. 2024 · 2024
Closest in time.
Large language models: A survey
Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. 2024 · 2024
Closest in time.
Language models with conformal factuality guarantees
Christopher Mohri and Tatsunori Hashimoto. 2024 · 2024
Closest in time.
James F Mullen Jr and Dinesh Manocha. 2024 · 2024
Closest in time.
Are Large Language Models More Honest in Their Probabilistic or Verbalized Confidence?
Shiyu Ni, Keping Bi, Lulu Yu, and Jiafeng Guo. 2024 · 2024
Closest in time.
Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities
Alexander Nikitin, Jannik Kossen, Yarin Gal, and Pekka Marttinen. 2024 · 2024
Closest in time.
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs
Ruijia Niu, Dongxia Wu, Rose Yu, and Yi-An Ma. 2024 · 2024
Closest in time.
Text clustering with LLM embeddings
Alina Petukhova, Joao P Matos-Carvalho, and Nuno Fachada. 2024 · 2024
Closest in time.
Semantic Density: Uncertainty Quantification in Semantic Space for Large Language Models
Xin Qiu and Risto Miikkulainen. 2024 · 2024
Closest in time.
A practical review of mechanistic interpretability for transformer-based language models
Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. 2024 · 2024
Closest in time.
Explore until Confident: Efficient Exploration for Embodied Question Answering
Allen Z Ren, Jaden Clark, Anushri Dixit, Masha Itkina, Anirudha Majumdar, and Dorsa Sadigh. 2024 · 2024
Closest in time.
CONFLARE: CONFormal LArge language model REtrieval
Pouria Rouzrokh, Shahriar Faghani, Cooper U Gamble, Moein Shariatnia, and Bradley J Erickson. 2024 · 2024
Closest in time.
Calibration and correctness of language models for code
Claudio Spiess, David Gros, Kunal Suresh Pai, Michael Pradel, Md Rafiqul Islam Rabin, Amin Alipour, Susmit Jha, Prem Devanbu, and Toufique Ahmed. 2024 · 2024
Closest in time.
Linguistic Obfuscation Attacks and Large Language Model Uncertainty. In Proceedings of the 1st Workshop on Uncertainty-Aware NLP (UncertaiNLP 2024) . 35–40
Sebastian Steindl, Ulrich Schäfer, Bernd Ludwig, and Patrick Levi. 2024 · 2024
Closest in time.
LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Models
Elias Stengel-Eskin, Peter Hase, and Mohit Bansal. 2024 · 2024
Closest in time.
Roformer: Enhanced transformer with rotary position embedding
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024a · 2024
Closest in time.
Api is enough: Conformal prediction for large language models without logit-access
Jiayuan Su, Jing Luo, Hongwei Wang, and Lu Cheng. 2024b · 2024
Closest in time.
Xingpeng Sun, Yiran Zhang, Xindi Tang, Amrit Singh Bedi, and Aniket Bera. 2024b · 2024
Closest in time.
ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability
Zhongxiang Sun, Xiaoxue Zang, Kai Zheng, Yang Song, Jun Xu, Xiao Zhang, Weijie Yu, and Han Li. 2024a · 2024
Closest in time.
An examination of the use of large language models to aid analysis of textual data
Robert H Tai, Lillian R Bentley, Xin Xia, Jason M Sitt, Sarah C Fankhauser, Ana M Chicas-Mosier, and Barnas G Monteith. 2024 · 2024
Closest in time.
An Evaluation of Estimative Uncertainty in Large Language Models
Zhisheng Tang, Ke Shen, and Mayank Kejriwal. 2024 · 2024
Closest in time.
Quantifying uncertainty in natural language explanations of large language models. In International Conference on Artificial Intelligence and Statistics . PMLR, 1072–1080
Sree Harsha Tanneru, Chirag Agarwal, and Himabindu Lakkaraju. 2024 · 2024
Closest in time.
When to Trust LLMs: Aligning Confidence with Response Quality
Shuchang Tao, Liuyi Yao, Hanxing Ding, Yuexiang Xie, Qi Cao, Fei Sun, Jinyang Gao, Huawei Shen, and Bolin Ding. 2024 · 2024
Closest in time.
Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, et al · 2024
Closest in time.
Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations
Christian Tomani, Kamalika Chaudhuri, Ivan Evtimov, Daniel Cremers, and Mark Ibrahim. 2024 · 2024
Closest in time.
A comprehensive survey of hallucination mitigation techniques in large language models
SM Tonmoy, SM Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, and Amitava Das. 2024 · 2024
Closest in time.
Yao-Hung Hubert Tsai, Walter Talbott, and Jian Zhang. 2024 · 2024
Closest in time.
Calibrating Large Language Models Using Their Generations Only
Dennis Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, and Seong Joon Oh. 2024 · 2024
Closest in time.
Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
Roman Vashurin, Ekaterina Fadeeva, Artem Vazhentsev, Akim Tsvigun, Daniil Vasilev, Rui Xing, Abdelrahman Boda Sadallah, Lyudmila Rvanova, Sergey Petrakov, Alexander Panchenko, et al · 2024
Closest in time.
Artem Vazhentsev, Ekaterina Fadeeva, Rui Xing, Alexander Panchenko, Preslav Nakov, Timothy Baldwin, Maxim Panov, and Artem Shelmanov. 2024 · 2024
Closest in time.
Epistemic Uncertainty Quantification For Pre-Trained Neural Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 11052–11061
Hanjing Wang and Qiang Ji. 2024 · 2024
Closest in time.
Safe Task Planning for Language-Instructed Multi-Robot Systems using Conformal Prediction
Jun Wang, Guocheng He, and Yiannis Kantaros. 2024c · 2024
Closest in time.
Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation
Yiming Wang, Pei Zhang, Baosong Yang, Derek F Wong, and Rui Wang. 2024d · 2024
Closest in time.
CLUE: Concept-Level Uncertainty Estimation for Large Language Models
Yu-Hsiang Wang, Andrew Bai, Che-Ping Tsai, and Cho-Jui Hsieh. 2024a · 2024
Closest in time.
ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees
Zhiyuan Wang, Jinhao Duan, Lu Cheng, Yue Zhang, Qingni Wang, Hengtao Shen, Xiaofeng Zhu, Xiaoshuang Shi, and Kaidi Xu. 2024b · 2024
Closest in time.
Posterior Uncertainty Quantification in Neural Networks using Data Augmentation. In International Conference on Artificial Intelligence and Statistics . PMLR, 3376–3384
Luhuan Wu and Sinead A Williamson. 2024 · 2024
Closest in time.
SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales
Tianyang Xu, Shujin Wu, Shizhe Diao, Xiaoze Liu, Xingyao Wang, Yangyi Chen, and Jing Gao. 2024b · 2024
Closest in time.
Hallucination is inevitable: An innate limitation of large language models
Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. 2024a · 2024
Closest in time.
To Believe or Not to Believe Your LLM
Yasin Abbasi Yadkori, Ilja Kuzborskij, András György, and Csaba Szepesvári. 2024 · 2024
Closest in time.
Bayesian Low-rank Adaptation for Large Language Models
Adam X. Yang, Maxime Robeyns, Xi Wang, and Laurence Aitchison. 2024a · 2024
Closest in time.
Can We Trust LLMs? Mitigate Overconfidence Bias in LLMs through Knowledge Transfer
Haoyan Yang, Yixuan Wang, Xingyin Xu, Hanyuan Zhang, and Yirong Bian. 2024b · 2024
Closest in time.
Benchmarking llms via uncertainty quantification
Fanghua Ye, Mingming Yang, Jianhui Pang, Longyue Wang, Derek F Wong, Emine Yilmaz, Shuming Shi, and Zhaopeng Tu. 2024 · 2024
Closest in time.
Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?
Gal Yona, Roee Aharoni, and Mor Geva. 2024 · 2024
Closest in time.
Mechanisms of non-factual hallucinations in language models
Lei Yu, Meng Cao, Jackie Chi Kit Cheung, and Yue Dong. 2024 · 2024
Closest in time.
Uncertainty is Fragile: Manipulating Uncertainty in Large Language Models
Qingcheng Zeng, Mingyu Jin, Qinkai Yu, Zhenting Wang, Wenyue Hua, Zihao Zhou, Guangyan Sun, Yanda Meng, Shiqing Ma, Qifan Wang, et al · 2024
Closest in time.
LUQ: Long-text Uncertainty Quantification for LLMs
Caiqi Zhang, Fangyu Liu, Marco Basaldella, and Nigel Collier. 2024 · 2024
Closest in time.
Explainability for large language models: A survey
Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024a · 2024
Closest in time.
SAUP: Situation Awareness Uncertainty Propagation on LLM Agent
Qiwei Zhao, Xujiang Zhao, Yanchi Liu, Wei Cheng, Yiyou Sun, Mika Oishi, Takao Osaki, Katsushi Matsuda, Huaxiu Yao, and Haifeng Chen. 2024d · 2024
Closest in time.
Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models
Xinran Zhao, Hongming Zhang, Xiaoman Pan, Wenlin Yao, Dong Yu, Tongshuang Wu, and Jianshu Chen. 2024c · 2024
Closest in time.
Evaluating Uncertainty-based Failure Detection for Closed-Loop LLM Planners
Zhi Zheng, Qian Feng, Hang Li, Alois Knoll, and Jianxiang Feng. 2024 · 2024
Closest in time.
Scale alone does not improve mechanistic interpretability in vision models
Roland S Zimmermann, Thomas Klein, and Wieland Brendel. 2024 · 2024
Closest in time.