Fetching the paper…
Reading the bibliography…
When answering questions, LLMs can convey not only an answer, but a level of confidence about the answer being correct.
An optimum character recognition system using decision functions
Chi-Keung Chow · 1957
Earlier work this paper cites.
Unified pragmatic models for generating and following instructions
Daniel Fried, Jacob Andreas, and Dan Klein · 1963
Earlier work this paper cites.
Logic and Conversation
Herbert P Grice · 1975
Earlier work this paper cites.
Automation bias in intelligent time critical decision support systems
Mary Cummings · 2004
Earlier work this paper cites.
Angeliki Lazaridou, Anna Potapenko, and Olivier Tieleman · 2005
Earlier work this paper cites.
Predicting pragmatic reasoning in language games
Michael C Frank and Noah D Goodman · 2012
Earlier work this paper cites.
Obtaining well calibrated probabilities using Bayesian binning
Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht · 2015
Earlier work this paper cites.
Selective classification for deep neural networks
Yonatan Geifman and Ran El-Yaniv · 2017
Earlier work this paper cites.
On calibration of modern neural networks
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger · 2017
Earlier work this paper cites.
Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension
Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer · 2017
Earlier work this paper cites.
Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek · 2019
Earlier work this paper cites.
On the inference calibration of neural machine translation
Shuo Wang, Zhaopeng Tu, Shuming Shi, and Yang Liu · 2020
Earlier work this paper cites.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al · 2020
Cited alongside, same era.
Unsolved problems in ML safety
Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt · 2021
Cited alongside, same era.
TruthfulQA: Measuring how models mimic human falsehoods
Stephanie Lin, Jacob Hilton, and Owain Evans · 2021
Cited alongside, same era.
Calibrate your listeners! Robust communication-based training for pragmatic speakers
Rose Wang, Julia White, Jesse Mu, and Noah Goodman · 2021
Cited alongside, same era.
Language models (mostly) know what they know
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, et al · 2022
Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning · 2023
Later among the works it cites.
Generating pragmatic examples to train neural program synthesizers
Saujas Vaduguru, Daniel Fried, and Yewen Pu · 2023
Later among the works it cites.
R-tuning: Teaching large language models to refuse unknown questions
Hanning Zhang, Shizhe Diao, Yong Lin, Yi R Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, and Tong Zhang · 2023
Later among the works it cites.
Navigating the grey area: How expressions of uncertainty and overconfidence affect language models
Kaitlyn Zhou, Dan Jurafsky, and Tatsunori Hashimoto · 2023
Later among the works it cites.
Moral AI: And How We Get There
Jana Schaich Borg, Walter Sinnott-Armstrong, and Vincent Conitzer · 2024
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar · 2022
Cited alongside, same era.
Teaching models to express their uncertainty in words
Stephanie Lin, Jacob Hilton, and Owain Evans · 2022
Cited alongside, same era.
Reducing conversational agents’ overconfidence through linguistic calibration
Sabrina J Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau · 2022
Cited alongside, same era.
Factors influencing ChatGPT adoption for product research and information retrieval
Vinayaka Gude · 2023
Cited alongside, same era.
A survey of hallucination in large foundation models
Vipula Rawte, Amit Sheth, and Amitava Das · 2023
Cited alongside, same era.
Robots that ask for help: Uncertainty alignment for large language model planners
Allen Z Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, et al · 2023
Cited alongside, same era.
Calibrated interpretation: Confidence estimation in semantic parsing
Elias Stengel-Eskin and Benjamin Van Durme · 2023
Cited alongside, same era.
QLoRA: Efficient finetuning of quantized LLMs
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer · 2024
Closest in time.
Sunnie SY Kim, Q Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, and Jennifer Wortman Vaughan · 2024
Closest in time.
Few-shot recalibration of language models
Xiang Lisa Li, Urvashi Khandelwal, and Kelvin Guu · 2024
Closest in time.
LLM evaluators recognize and favor their own generations
Arjun Panickssery, Samuel R Bowman, and Shi Feng · 2024
Closest in time.
Direct preference optimization: Your language model is secretly a reward model
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn · 2024
Closest in time.
Calibrating large language models using their generations only
Dennis Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, and Seong Joon Oh · 2024
Closest in time.
Relying on the unreliable: The impact of language models’ reluctance to express uncertainty
Kaitlyn Zhou, Jena D Hwang, Xiang Ren, and Maarten Sap · 2024
Closest in time.