Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations

Understand

LLMs often adopt an assertive language style also when making false claims.

• Such ``overconfident hallucinations'' mislead users and erode trust.
• Achieving the ability to express in language the actual degree of uncertainty around a claim is therefore of great importance.
• We find that ``verbal uncertainty'' is governed by a single linear feature in the representation space of LLMs, and show that this has only moderate correlation with the actual ``semantic uncertainty'' of the model.