CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generation

Understand

Code generation is a longstanding challenge, aiming to generate a code snippet based on a natural language description.

• Usually, expensive text-code paired data is essential for training a code generation model.
• Recently, thanks to the success of pre-training techniques, large language models are trained on large-scale unlabelled code corpora and perform well in code generation.
• In this paper, we investigate how to leverage an unlabelled code corpus to train a model for library-oriented code generation.