Universal Self-Consistency for Large Language Model Generation

Understand

Self-consistency with chain-of-thought prompting (CoT) has demonstrated remarkable performance gains on various challenging tasks, by utilizing multiple reasoning paths sampled from large language models (LLMs).

• However, self-consistency relies on the answer extraction process to aggregate multiple solutions, which is not applicable to free-form answers.
• In this work, we propose Universal Self-Consistency (USC), which leverages LLMs themselves to select the most consistent answer among multiple candidates.
• We evaluate USC on a variety of benchmarks, including mathematical reasoning, code generation, long-context summarization, and open-ended question answering.