A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice

A comprehensive guide to bayesian convolutional neural network with variational inference

Original

Kumar Shridhar, Felix Laumann, and Marcus Liwicki. 2019 · 1901

Earlier work this paper cites.

On information and sufficiency

S. Kullback and R. A. Leibler. 1951 · 1951

Earlier work this paper cites.

Dropout: a simple way to prevent neural networks from overfitting

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014 · 1958

Earlier work this paper cites.

Information theory

Robert B Ash. 1965 · 1965

Earlier work this paper cites.

The evolution of medical uncertainty

Renee C Fox. 1980 · 1980

Earlier work this paper cites.

Active learning with statistical models

David A Cohn, Zoubin Ghahramani, and Michael I Jordan. 1996 · 1996

Earlier work this paper cites.

Are humans good intuitive statisticians after all? rethinking some conclusions from the literature on judgment under uncertainty

Leda Cosmides and John Tooby. 1996 · 1996

Earlier work this paper cites.

An introduction to variational methods for graphical models

Michael I Jordan, Zoubin Ghahramani, Tommi S Jaakkola, and Lawrence K Saul. 1999 · 1999

Earlier work this paper cites.

Batchensemble: An alternative approach to efficient ensemble and lifelong learning

Original

Yeming Wen, Dustin Tran, and Jimmy Ba. 2020 · 2002

Earlier work this paper cites.

Simple and principled uncertainty estimation with deterministic deep learning via distance awareness

Original

Jeremiah Zhe Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax-Weiss, and Balaji Lakshminarayanan. 2020 · 2006

Earlier work this paper cites.

Aleatory or epistemic? does it matter?

Armen Der Kiureghian and Ove Ditlevsen. 2009 · 2009

Earlier work this paper cites.

Practical variational inference for neural networks

Alex Graves. 2011 · 2011

Earlier work this paper cites.

Weight uncertainty in neural network

Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015 · 2015

Earlier work this paper cites.

Tolerating uncertainty—the next medical revolution?

A Simpkin and Richard Schwartzstein. 2016 · 2016

Earlier work this paper cites.

Improving output uncertainty estimation and generalization in deep learning via neural network gaussian processes

Original

Tomoharu Iwata and Zoubin Ghahramani. 2017 · 2017

Earlier work this paper cites.

Simple and scalable predictive uncertainty estimation using deep ensembles

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017 · 2017

Earlier work this paper cites.

Deep active learning for named entity recognition

Original

Yanyao Shen, Hyokun Yun, Zachary C Lipton, Yakov Kronrod, and Animashree Anandkumar. 2017 · 2017

Earlier work this paper cites.

Predictive uncertainty estimation via prior networks

Original

Andrey Malinin and Mark Gales. 2018 · 2018

Earlier work this paper cites.

Quantifying uncertainties in natural language processing tasks

Original

Yijun Xiao and William Yang Wang. 2018 · 2018

Earlier work this paper cites.

Modeling epistemic and aleatoric uncertainty with bayesian neural networks and latent variables

Stefan Depeweg. 2019 · 2019

Earlier work this paper cites.

Noise contrastive priors for functional uncertainty

Original

Danijar Hafner, Dustin Tran, Timothy Lillicrap, Alex Irpan, and James Davidson. 2019 · 2019

Earlier work this paper cites.

Uncertainty estimation in deep learning with application to spoken language assessment

Andrey Malinin. 2019 · 2019

Earlier work this paper cites.

Comparing ensemble strategies for deep learning: An application to facial expression recognition

Alessandro Renda, Marco Barsacchi, Alessio Bechini, and Francesco Marcelloni. 2019 · 2019

Earlier work this paper cites.

Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods

Eyke Hüllermeier and Willem Waegeman. 2021 · 2021

Earlier work this paper cites.

On the importance of effectively adapting pretrained language models for active learning

Original

Katerina Margatina, Loic Barrault, and Nikolaos Aletras. 2021 · 2021

Cited alongside, same era.

Small-text: Active learning for text classification in python

Original

Christopher Schröder, Lydia Müller, Andreas Niekler, and Martin Potthast. 2021 · 2021

Cited alongside, same era.

On hallucination and predictive uncertainty in conditional language generation

Original

Yijun Xiao and William Yang Wang. 2021 · 2021

Cited alongside, same era.

Language models (mostly) know what they know

Original

Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022 · 2022

Cited alongside, same era.

In chatgpt we trust? measuring and characterizing the reliability of chatgpt

Original

Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. 2023 · 2023

Later among the works it cites.

Llamas know what gpts don’t show: Surrogate models for confidence estimation

Original

Vaishnavi Shrivastava, Percy Liang, and Ananya Kumar. 2023 · 2023

Later among the works it cites.

Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback

Original

Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D. Manning. 2023 · 2023

Later among the works it cites.

Hybrid uncertainty quantification for selective text classification in ambiguous tasks

Artem Vazhentsev, Gleb Kuzmin, Akim Tsvigun, Alexander Panchenko, Maxim Panov, Mikhail Burtsev, and Artem Shelmanov. 2023 · 2023

Later among the works it cites.