Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models

Understand

As the use of Large Language Models (LLMs) becomes more widespread, understanding their self-evaluation of confidence in generated responses becomes increasingly important as it is integral to the reliability of the output of these models.

• We introduce the concept of Confidence-Probability Alignment, that connects an LLM's internal confidence, quantified by token probabilities, to the confidence conveyed in the model's response when explicitly asked about its certainty.
• Using various datasets and prompting techniques that encourage model introspection, we probe the alignment between models' internal and expressed confidence.
• These techniques encompass using structured evaluation scales to rate confidence, including answer options when prompting, and eliciting the model's confidence level for outputs it does not recognize as its own.