Finetuning Large Language Models for Vulnerability Detection

Understand

This paper presents the results of finetuning large language models (LLMs) for the task of detecting vulnerabilities in source code.

• We leverage WizardCoder, a recent improvement of the state-of-the-art LLM StarCoder, and adapt it for vulnerability detection through further finetuning.
• To accelerate training, we modify WizardCoder's training procedure, also we investigate optimal training regimes.
• For the imbalanced dataset with many more negative examples than positive, we also explore different techniques to improve classification performance.