2024

Pros and Cons! Evaluating ChatGPT on Software Vulnerability

Yin, Xin

Understand

This paper proposes a pipeline for quantitatively evaluating interactive LLMs such as ChatGPT using publicly available dataset.

We carry out an extensive technical evaluation of ChatGPT using Big-Vul covering five different common software vulnerability tasks.
We evaluate the multitask and multilingual aspects of ChatGPT based on this dataset.
We found that the existing state-of-the-art methods are generally superior to ChatGPT in software vulnerability detection.

Built on

Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 1901
Earlier work this paper cites.
Rouge: A package for automatic evaluation of summaries. In Text summarization branches out . 74–81
Chin-Yew Lin. 2004 · 2004
Earlier work this paper cites.
Deep reinforcement learning from human preferences
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017 · 2017
Earlier work this paper cites.
Proximal policy optimization algorithms
Original
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017 · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
The effect of common vulnerability scoring system metrics on vulnerability exploit delay. In 2018 Sixth International Symposium on Computing and Networking (CANDAR) . IEEE, 1–10
Andrew Feutrill, Dinesha Ranathunga, Yuval Yarom, and Matthew Roughan. 2018 · 2018
Earlier work this paper cites.
Review into state of the art of vulnerability assessment using artificial intelligence
Saad Khan and Simon Parkinson. 2018 · 2018
Earlier work this paper cites.
Vuldeepecker: A deep learning-based system for vulnerability detection. In Proceedings of the 25th Annual Network and Distributed System Security Symposium
Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang, Zhijun Deng, and Yuyi Zhong. 2018 · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al · 2018
Earlier work this paper cites.
A multi-target approach to estimate software vulnerability characteristics and severity scores
Georgios Spanos and Lefteris Angelis. 2018 · 2018
Earlier work this paper cites.
Sequencer: Sequence-to-sequence learning for end-to-end program repair
Zimin Chen, Steve James Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, and Martin Monperrus. 2019 · 2019
Earlier work this paper cites.
Smoke: scalable path-sensitive memory leak detection for millions of lines of code. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) . IEEE, 72–82
Gang Fan, Rongxin Wu, Qingkai Shi, Xiao Xiao, Jinguo Zhou, and Charles Zhang. 2019 · 2019
Earlier work this paper cites.
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019 · 2019
Earlier work this paper cites.
Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. In In Proceedings of the 33rd International Conference on Neural Information Processing Systems . 10197–10207
Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019 · 2019
Earlier work this paper cites.
Fine-tuning language models from human preferences
Original
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019 · 2019
Earlier work this paper cites.
A C/C++ code vulnerability dataset with code changes and CVE summaries. In Proceedings of the 17th International Conference on Mining Software Repositories . 508–512
Jiahao Fan, Yi Li, Shaohua Wang, and Tien N Nguyen. 2020 · 2020
Earlier work this paper cites.
Codebert: A pre-trained model for programming and natural languages
Original
Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al · 2020
Earlier work this paper cites.
Graphcodebert: Pre-training code representations with data flow
Original
Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al · 2020
Earlier work this paper cites.
Predicting missing information of key aspects in vulnerability reports
Original
Hao Guo, Zhenchang Xing, and Xiaohong Li. 2020b · 2020
Earlier work this paper cites.

Similar

PCA: memory leak detection using partial call-path analysis. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . 1621–1625
Wen Li, Haipeng Cai, Yulei Sui, and David Manz. 2020 · 2020
Cited alongside, same era.
Unified pre-training for program understanding and generation
Original
Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021 · 2021
Cited alongside, same era.
CVEfixes: automated collection of vulnerabilities and their fixes from open-source software. In Proceedings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering . 30–39
Guru Bhandari, Amara Naseer, and Leon Moonen. 2021 · 2021
Cited alongside, same era.
Deep learning based vulnerability detection: Are we there yet
Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray. 2021 · 2021
Cited alongside, same era.
Key aspects augmentation of vulnerability description based on multiple security databases. In 2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC) . IEEE, 1020–1025
Hao Guo, Zhenchang Xing, Sen Chen, Xiaohong Li, Yude Bai, and Hu Zhang. 2021 · 2021
Cited alongside, same era.
A survey on data-driven software vulnerability assessment and prioritization
Triet HM Le, Huaming Chen, and M Ali Babar. 2021a · 2021
Cited alongside, same era.
Deepcva: Automated commit-level vulnerability assessment with deep multi-task learning. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) . IEEE, 717–729
Triet Huynh Minh Le, David Hin, Roland Croft, and M Ali Babar. 2021b · 2021
Cited alongside, same era.
Vuldeelocator: a deep learning-based fine-grained vulnerability detector
Zhen Li, Deqing Zou, Shouhuai Xu, Zhaoxuan Chen, Yawei Zhu, and Hai Jin. 2021b · 2021
Cited alongside, same era.
Sysevr: A framework for using deep learning to detect software vulnerabilities
Zhen Li, Deqing Zou, Shouhuai Xu, Hai Jin, Yawei Zhu, and Zhaoxuan Chen. 2021c · 2021
Cited alongside, same era.
Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation
Original
Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. 2021 · 2021
Cited alongside, same era.
Atvhunter: Reliable version detection of third-party libraries for vulnerability identification in android applications. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) . IEEE, 1695–1707
Xian Zhan, Lingling Fan, Sen Chen, Feng We, Tianming Liu, Xiapu Luo, and Yang Liu. 2021 · 2021
Cited alongside, same era.
A syntax-guided edit decoder for neural program repair. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . 341–353
Qihao Zhu, Zeyu Sun, Yuan-an Xiao, Wenjie Zhang, Kang Yuan, Yingfei Xiong, and Lu Zhang. 2021 · 2021
Cited alongside, same era.

Then

The Best of Both Worlds: Integrating Semantic Features with Expert Features for Defect Prediction and Localization. In Proceedings of the 2022 30th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . ACM, 672–683
Chao Ni, Wei Wang, Kaiwen Yang, Xin Xia, Kui Liu, and David Lo. 2022a · 2022
Later among the works it cites.
Defect Identification, Categorization, and Repair: Better Together
Original
Chao Ni, Kaiwen Yang, Xin Xia, David Lo, Xiang Chen, and Xiaohu Yang. 2022b · 2022
Later among the works it cites.
ChatGPT: Optimizing Language Models for Dialogue. (2022)
OpenAI. 2022 · 2022
Later among the works it cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Later among the works it cites.
Generating informative CVE description from ExploitDB posts by extractive summarization
Jiamou Sun, Zhenchang Xing, Hao Guo, Deheng Ye, Xiaohong Li, Xiwei Xu, and Liming Zhu. 2022 · 2022
Later among the works it cites.
VulCNN: An Image-inspired Scalable Vulnerability Detection System
Yueming Wu, Deqing Zou, Shihan Dou, Wei Yang, Duo Xu, and Hai Jin. 2022 · 2022
Later among the works it cites.
Program Repair: Automated vs. Manual
Original
Quanjun Zhang, Yuan Zhao, Weisong Sun, Chunrong Fang, Ziyuan Wang, and Lingming Zhang. 2022 · 2022
Later among the works it cites.
Replication
2023 · 2023
Later among the works it cites.
A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity
Original
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al · 2023
Later among the works it cites.
Introducing ChatGPT and Whisper APIs
chatgptendpoint. 2023 · 2023
Later among the works it cites.
Chatgpt goes to law school
Jonathan H Choi, Kristin E Hickman, Amy Monahan, and Daniel Schwarcz. 2023 · 2023
Later among the works it cites.
Friend or foe? teachers debate chatgpt
Yoav Goldberg. 2023a · 2023
Later among the works it cites.
Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023 · 2023
Later among the works it cites.
Common Vulnerabilities and Exposures (CVE)
Corporation MITRE. 2023 · 2023
Later among the works it cites.
FVA: Assessing Function-Level Vulnerability by Integrating Flow-Sensitive Structure and Code Statement Semantic. In 2023 IEEE/ACM 31st International Conference on Program Comprehension (ICPC) . IEEE, 339–350
Chao Ni, Liyu Shen, Wei Wang, Xiang Chen, Xin Yin, and Lexiao Zhang. 2023a · 2023
Later among the works it cites.
ChatGPT and other large language models are double-edged swords
Yiqiu Shen, Laura Heacock, Jonathan Elias, Keith D Hentel, Beatriu Reig, George Shih, and Linda Moy. 2023 · 2023
Later among the works it cites.
Best practices for prompt engineering with OpenAI API
Jessica Shieh. 2023 · 2023
Later among the works it cites.
securityFocus
Symantec. 2023 · 2023
Later among the works it cites.
Multitask-based Evaluation of Open-Source LLM on Software Vulnerability
Original
Xin Yin and Chao Ni. 2024 · 2024
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…