Fetching the paper…

Reading the bibliography…

2024

Cycles of Thought: Measuring LLM Confidence through Stable Explanations

Becker, Evan, Soatto, Stefano

Original
PDF
Post on X

Understand

In many high-risk machine learning applications it is essential for a model to indicate when it is uncertain about a prediction.

While large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, their overconfidence in incorrect responses is still a well-documented failure mode.
Traditional methods for ML uncertainty quantification can be difficult to directly adapt to LLMs due to the computational cost of implementation and closed-source nature of many models.
A variety of black-box methods have recently been proposed, but these often rely on heuristics such as self-verbalized confidence.

Cycles of Thought: Measuring LLM Confidence through Stable Explanations · Around

Built on

Three approaches to the quantitative definition ofinformation’
A. N. Kolmogorov · 1965
Earlier work this paper cites.
Transductive inference for text classification using support vector machines
T. Joachims et al · 1999
Earlier work this paper cites.
A tutorial on conformal prediction
G. Shafer and V. Vovk · 2008
Earlier work this paper cites.
Bayesian learning for neural networks , volume 118
R. M. Neal · 2012
Earlier work this paper cites.
Obtaining well calibrated probabilities using bayesian binning
M. P. Naeini, G. Cooper, and M. Hauskrecht · 2015
Earlier work this paper cites.
A baseline for detecting misclassified and out-of-distribution examples in neural networks
Original
D. Hendrycks and K. Gimpel · 2016
Earlier work this paper cites.
Reasoning about uncertainty
J. Y. Halpern · 2017
Earlier work this paper cites.
Predictive uncertainty estimation via prior networks
A. Malinin and M. Gales · 2018
Earlier work this paper cites.
Commonsenseqa: A question answering challenge targeting commonsense knowledge
Original
A. Talmor, J. Herzig, N. Lourie, and J. Berant · 2018
Earlier work this paper cites.
Language models are few-shot learners
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al · 2020
Earlier work this paper cites.
A baseline for few-shot image classification
G. S. Dhillon, P. Chaudhari, A. Ravichandran, and S. Soatto · 2020
Earlier work this paper cites.
Revisiting the evaluation of uncertainty estimation and its application to explore model complexity-uncertainty trade-off
Y. Ding, J. Liu, J. Xiong, and Y. Shi · 2020
Earlier work this paper cites.
Measuring massive multitask language understanding
Original
D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt · 2020
Earlier work this paper cites.
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharya, et al · 2021
Earlier work this paper cites.
What disease does this patient have? a large-scale open domain question answering dataset from medical exams
D. Jin, E. Pan, N. Oufattole, W.-H. Weng, H. Fang, and P. Szolovits · 2021
Earlier work this paper cites.
Truthfulqa: Measuring how models mimic human falsehoods
Original
S. Lin, J. Hilton, and O. Evans · 2021
Earlier work this paper cites.
On hallucination and predictive uncertainty in conditional language generation
Original
Y. Xiao and W. Y. Wang · 2021
Earlier work this paper cites.

Similar

Generalized out-of-distribution detection: A survey
Original
J. Yang, K. Zhou, Y. Li, and Z. Liu · 2021
Cited alongside, same era.
Discovering latent knowledge in language models without supervision
Original
C. Burns, H. Ye, D. Klein, and J. Steinhardt · 2022
Cited alongside, same era.
Why can gpt learn in-context? language models implicitly perform gradient descent as meta-optimizers
Original
D. Dai, Y. Sun, L. Dong, Y. Hao, S. Ma, Z. Sui, and F. Wei · 2022
Cited alongside, same era.
Hands-on bayesian neural networks—a tutorial for deep learning users
L. V. Jospin, H. Laga, F. Boussaid, W. Buntine, and M. Bennamoun · 2022
Cited alongside, same era.
Language models (mostly) know what they know
Original

Then

Look before you leap: An exploratory study of uncertainty measurement for large language models
Original
Y. Huang, J. Song, Z. Wang, H. Chen, and L. Ma · 2023
Later among the works it cites.
Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation
Original
L. Kuhn, Y. Gal, and S. Farquhar · 2023
Later among the works it cites.
Conformal prediction with large language models for multi-choice question answering
Original
B. Kumar, C. Lu, G. Gupta, A. Palepu, D. Bellamy, R. Raskar, and A. Beam · 2023
Later among the works it cites.
Llamas know what gpts don’t show: Surrogate models for confidence estimation
Original
V. Shrivastava, P. Liang, and A. Kumar · 2023
Later among the works it cites.
Taming AI bots: Controllability of neural states in large language models

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

S. Kadavath, T. Conerly, A. Askell, T. Henighan, D. Drain, E. Perez, N. Schiefer, Z. Hatfield-Dodds, N. DasSarma, E. Tran-Johnson, et al · 2022

Cited alongside, same era.

Self-consistency improves chain of thought reasoning in language models

X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, and D. Zhou · 2022

Cited alongside, same era.

Chain-of-thought prompting elicits reasoning in large language models

J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al · 2022

Cited alongside, same era.

Gpt-4 technical report

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al · 2023

Cited alongside, same era.

The internal state of an llm knows when its lying

A. Azaria and T. Mitchell · 2023

Cited alongside, same era.

Uncertainty in natural language generation: From theory to applications

J. Baan, N. Daheim, E. Ilia, D. Ulmer, H.-S. Li, R. Fernández, B. Plank, R. Sennrich, C. Zerva, and W. Aziz · 2023

Cited alongside, same era.

A survey on evaluation of large language models

Y. Chang, X. Wang, J. Wang, Y. Wu, L. Yang, K. Zhu, H. Chen, X. Yi, C. Wang, Y. Wang, et al · 2023

Cited alongside, same era.

Quantifying uncertainty in answers from any language model via intrinsic and extrinsic confidence assessment

J. Chen and J. Mueller · 2023

Cited alongside, same era.

S. Soatto, P. Tabuada, P. Chaudhari, and T. Y. Liu · 2023

Later among the works it cites.

Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback

K. Tian, E. Mitchell, A. Zhou, A. Sharma, R. Rafailov, H. Yao, C. Finn, and C. D. Manning · 2023

Later among the works it cites.

Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms

M. Xiong, Z. Hu, X. Lu, Y. Li, J. Fu, J. He, and B. Hooi · 2023

Later among the works it cites.

Do large language models know what they don’t know?

Z. Yin, Q. Sun, Q. Guo, J. Wu, X. Qiu, and X. Huang · 2023

Later among the works it cites.

Towards better chain-of-thought prompting strategies: A survey

Z. Yu, L. He, Z. Wu, X. Dai, and J. Chen · 2023

Later among the works it cites.

Siren’s song in the ai ocean: a survey on hallucination in large language models

Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen, et al · 2023

Later among the works it cites.

Batch calibration: Rethinking calibration for in-context learning and prompt engineering

H. Zhou, X. Wan, L. Proleev, D. Mincu, J. Chen, K. Heller, and S. Roy · 2023

Later among the works it cites.

Think twice before assure: Confidence estimation for large language models through reflection on multiple answers

M. Li, W. Wang, F. Feng, F. Zhu, Q. Wang, and T.-S. Chua · 2024

Closest in time.

Meaning representations from trajectories in autoregressive models

T. Y. Liu, M. Trager, A. Achille, P. Perera, L. Zancato, and S. Soatto · 2024

Closest in time.

Minds versus machines: Rethinking entailment verification with language models

S. Sanyal, T. Xiao, J. Liu, W. Wang, and X. Ren · 2024

Closest in time.

The calibration gap between model and human confidence in large language models

M. Steyvers, H. Tejeda, A. Kumar, C. Belem, S. Karny, X. Hu, L. Mayer, and P. Smyth · 2024

Closest in time.

Language models don’t always say what they think: unfaithful explanations in chain-of-thought prompting

M. Turpin, J. Michael, E. Perez, and S. Bowman · 2024

Closest in time.

Fact-and-reflection (far) improves confidence calibration of large language models

X. Zhao, H. Zhang, X. Pan, W. Yao, D. Yu, T. Wu, and J. Chen · 2024

Closest in time.