From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models

Understand

One of the most striking findings in modern research on large language models (LLMs) is that scaling up compute during training leads to better results.

• However, less attention has been given to the benefits of scaling compute during inference.
• This survey focuses on these inference-time approaches.
• We explore three areas under a unified mathematical formalism: token-level generation algorithms, meta-generation algorithms, and efficient generation.