Inference-Time Language Model Alignment via Integrated Value Guidance

Understand

Large language models are typically fine-tuned to align with human preferences, but tuning large models is computationally intensive and complex.

• In this work, we introduce $\textit{Integrated Value Guidance}$ (IVG), a method that uses implicit and explicit value functions to guide language model decoding at token and chunk-level respectively, efficiently aligning large language models purely at inference time.
• This approach circumvents the complexities of direct fine-tuning and outperforms traditional methods.
• Empirically, we demonstrate the versatility of IVG across various tasks.