Multi-task Learning based Pre-trained Language Model for Code Completion

Understand

Code completion is one of the most useful features in the Integrated Development Environments (IDEs), which can accelerate software development by suggesting the next probable token based on the contextual code in real-time.

• Recent studies have shown that statistical language modeling techniques can improve the performance of code completion tools through learning from large-scale software repositories.
• However, these models suffer from two major drawbacks: a) Existing research uses static embeddings, which map a word to the same vector regardless of its context.
• The differences in the meaning of a token in varying contexts are lost when each token is associated with a single representation; b) Existing language model based code completion models perform poor on completing identifiers, and the type information of the identifiers is ignored in most of these models.