Scalable Best-of-N Selection for Large Language Models via Self-Certainty

Understand

Best-of-N selection is a key technique for improving the reasoning performance of Large Language Models (LLMs) through increased test-time computation.

• Current state-of-the-art methods often employ computationally intensive reward models for response evaluation and selection.
• Reward-free alternatives, like self-consistency and universal self-consistency, are limited in their ability to handle open-ended generation tasks or scale effectively.
• To address these limitations, we propose self-certainty, a novel and efficient metric that leverages the inherent probability distribution of LLM outputs to estimate response quality without requiring external reward models.