Planning with Large Language Models for Code Generation

Understand

Existing large language model-based code generation pipelines typically use beam search or sampling algorithms during the decoding process.

• Although the programs they generate achieve high token-matching-based scores, they often fail to compile or generate incorrect outputs.
• The main reason is that conventional Transformer decoding algorithms may not be the best choice for code generation.
• In this work, we propose a novel Transformer decoding algorithm, Planning-Guided Transformer Decoding (PG-TD), that uses a planning algorithm to do lookahead search and guide the Transformer to generate better programs.