Probing Pretrained Models of Source Code

Understand

Deep learning models are widely used for solving challenging code processing tasks, such as code generation or code summarization.

• Traditionally, a specific model architecture was carefully built to solve a particular code processing task.
• However, recently general pretrained models such as CodeBERT or CodeT5 have been shown to outperform task-specific models in many applications.
• While pretrained models are known to learn complex patterns from data, they may fail to understand some properties of source code.