Bridging Pre-trained Models and Downstream Tasks for Source Code Understanding

Understand

With the great success of pre-trained models, the pretrain-then-finetune paradigm has been widely adopted on downstream tasks for source code understanding.

• However, compared to costly training a large-scale model from scratch, how to effectively adapt pre-trained models to a new task has not been fully explored.
• In this paper, we propose an approach to bridge pre-trained models and code-related tasks.
• We exploit semantic-preserving transformation to enrich downstream data diversity, and help pre-trained models learn semantic features invariant to these semantically equivalent transformations.