Comparison of Static Application Security Testing Tools and Large Language Models for Repo-level Vulnerability Detection

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 1901

Earlier work this paper cites.

Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning

Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. 2022 · 1965

Earlier work this paper cites.

To what extent could we detect field defects? an empirical study of false negatives in static bug finding tools. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering . 50–59

Ferdian Thung, Lucia, David Lo, Lingxiao Jiang, Foyzur Rahman, and Premkumar T Devanbu. 2012 · 2012

Earlier work this paper cites.

On the capability of static code analysis to detect security vulnerabilities

Katerina Goseva-Popstojanova and Andrei Perhinschi. 2015 · 2015

Earlier work this paper cites.

Juliet Test Suite

National Institute of Standards and Technology. 2017 · 2017

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Large-scale analysis of framework-specific exceptions in android apps. In Proceedings of the 40th International Conference on Software Engineering . 408–419

Lingling Fan, Ting Su, Sen Chen, Guozhu Meng, Yang Liu, Lihua Xu, Geguang Pu, and Zhendong Su. 2018 · 2018

Earlier work this paper cites.

How many of all bugs do we find? a study of static bug detectors. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering . 317–328

Andrew Habib and Michael Pradel. 2018 · 2018

Earlier work this paper cites.

Analyzing the analyzers: Flowdroid/iccta, amandroid, and droidsafe. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis . 176–186

Lina Qiu, Yingying Wang, and Julia Rubin. 2018 · 2018

Earlier work this paper cites.

Bugs in the wild: examining the effectiveness of static analyzers at finding real-world bugs. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . 980–982

David A Tomassi. 2018 · 2018

Earlier work this paper cites.

An empirical study of security warnings from static application security testing tools

Bushra Aloraini, Meiyappan Nagappan, Daniel M German, Shinpei Hayashi, and Yoshiki Higo. 2019 · 2019

Earlier work this paper cites.

Implementation analysis of open-source Static analysis tools for detecting security vulnerabilities

Sindre Beba and Magnus Melseth Karlsen. 2019 · 2019

Earlier work this paper cites.

Deepjit: an end-to-end deep learning framework for just-in-time defect prediction. In 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR) . IEEE, 34–45

Thong Hoang, Hoa Khanh Dam, Yasutaka Kamei, David Lo, and Naoyasu Ubayashi. 2019 · 2019

Earlier work this paper cites.

Evaluation of open-source IDE plugins for detecting security vulnerabilities. In Proceedings of the 23rd International Conference on Evaluation and Assessment in Software Engineering . 200–209

Jingyue Li, Sindre Beba, and Magnus Melseth Karlsen. 2019 · 2019

Earlier work this paper cites.

An empirical study on combining diverse static analysis tools for web security vulnerabilities based on development scenarios

Paulo Nunes, Ibéria Medeiros, José Fonseca, Nuno Neves, Miguel Correia, and Marco Vieira. 2019 · 2019

Earlier work this paper cites.

Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks

Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019 · 2019

Earlier work this paper cites.

An empirical assessment of security risks of global android banking apps. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering . 1310–1322

Sen Chen, Lingling Fan, Guozhu Meng, Ting Su, Minhui Xue, Yinxing Xue, Yang Liu, and Lihua Xu. 2020 · 2020

Earlier work this paper cites.

AC/C++ code vulnerability dataset with code changes and CVE summaries. In Proceedings of the 17th International Conference on Mining Software Repositories . 508–512

Jiahao Fan, Yi Li, Shaohua Wang, and Tien N Nguyen. 2020 · 2020

Earlier work this paper cites.

CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020 (Findings of ACL, Vol. EMNLP 2020) , Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 1536–1547

Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020 · 2020

Earlier work this paper cites.

Graphcodebert: Pre-training code representations with data flow

Original

Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al · 2020

Earlier work this paper cites.

Static code analysis tools: A systematic literature review. In Ann. DAAAM Proc. Int. DAAAM Symp , Vol. 31. 565–573

Darko Stefanović, Danilo Nikolić, Dušanka Dakić, Ivana Spasojević, and Sonja Ristić. 2020 · 2020

Earlier work this paper cites.

Detecting security vulnerabilities with static analysis–A case study

Midya Alqaradaghi, Gregory Morse, and Tamás Kozsik. 2021 · 2021

Earlier work this paper cites.

CVEfixes: automated collection of vulnerabilities and their fixes from open-source software. In Proceedings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering . 30–39

Guru Bhandari, Amara Naseer, and Leon Moonen. 2021 · 2021

Earlier work this paper cites.

Deep learning based vulnerability detection: Are we there yet?

Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray. 2021 · 2021

Earlier work this paper cites.

An empirical study of rule-based and learning-based approaches for static application security testing. In Proceedings of the 15th ACM/IEEE international symposium on empirical software engineering and measurement (ESEM) . 1–12

Roland Croft, Dominic Newlands, Ziyu Chen, and M Ali Babar. 2021 · 2021

Cited alongside, same era.

Lora: Low-rank adaptation of large language models

Original

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021

Cited alongside, same era.

On the adoption of static analysis for software security assessment–A case study of an open-source e-government project

Anh Nguyen-Duc, Manh Viet Do, Quan Luong Hong, Kiem Nguyen Khac, and Anh Nguyen Quang. 2021 · 2021

Cited alongside, same era.

CrossVul: a cross-language vulnerability dataset with commit data. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . 1565–1569

Georgios Nikitopoulos, Konstantina Dritsa, Panos Louridas, and Dimitris Mitropoulos. 2021 · 2021

A comprehensive study on quality assurance tools for java. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis . 285–297

Han Liu, Sen Chen, Ruitao Feng, Chengwei Liu, Kaixuan Li, Zhengzi Xu, Liming Nie, Yang Liu, and Yixiang Chen. 2023 · 2023

Later among the works it cites.

CodeLlama

Meta. 2023 · 2023

Later among the works it cites.

PTLVD:Program Slicing and Transformer-based Line-level Vulnerability Detection System. In 23rd IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2023, Bogotá, Colombia, October 2-3, 2023 , Leon Moonen, Christian D. Newman, and Alessandra Gorla (Eds.). IEEE, 162–173

Tao Peng, Shixu Chen, Fei Zhu, Junwei Tang, Junping Liu, and Xinrong Hu. 2023 · 2023

Later among the works it cites.

Efficient methods for natural language processing: A survey

Marcos Treviso, Ji-Ung Lee, Tianchu Ji, Betty van Aken, Qingqing Cao, Manuel R Ciosici, Michael Hassid, Kenneth Heafield, Sara Hooker, Colin Raffel, et al · 2023

Later among the works it cites.