Type-Constrained Code Generation with Language Models

Understand

Large language models (LLMs) have achieved notable success in code generation.

• However, they still frequently produce uncompilable output because their next-token inference procedure does not model formal aspects of code.
• Although constrained decoding is a promising approach to alleviate this issue, it has only been applied to handle either domain-specific languages or syntactic features of general-purpose programming languages.
• However, LLMs frequently generate code with typing errors, which are beyond the domain of syntax and generally hard to adequately constrain.