Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG

Understand

Although LLMs have shown promising potential in vulnerability detection, this study reveals their limitations in distinguishing between vulnerable and similar-but-benign patched code (only 0.06 - 0.14 accuracy).

• It shows that LLMs struggle to capture the root causes of vulnerabilities during vulnerability detection.
• To address this challenge, we propose enhancing LLMs with multi-dimensional vulnerability knowledge distilled from historical vulnerabilities and fixes.
• We design a novel knowledge-level Retrieval-Augmented Generation framework Vul-RAG, which improves LLMs with an accuracy increase of 16% - 24% in identifying vulnerable and patched code.