LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning

C. Cadar, D. Dunbar, and D. Engler, “KLEE: Unassisted and automatic generation of High-Coverage tests for complex systems programs,” in Proc. USENIX OSDI , 2008

2008

Earlier work this paper cites.

T. Wang, T. Wei, G. Gu, and W. Zou, “Taintscope: A checksum-aware directed fuzzing tool for automatic software vulnerability detection,” in Proc. IEEE Symposium on Security and Privacy (S&P) , 2010

2010

Earlier work this paper cites.

S. Arzt, S. Rasthofer, C. Fritz, E. Bodden, A. Bartel, J. Klein, Y. Le Traon, D. Octeau, and P. McDaniel, “Flowdroid: precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for android apps,” in Proc. ACM PLDI , 2014

2014

Earlier work this paper cites.

Y. Shoshitaishvili, R. Wang, C. Salls, N. Stephens, M. Polino, A. Dutcher, J. Grosen, S. Feng, C. Hauser, C. Kruegel, and G. Vigna, “Sok: (state of) the art of war: Offensive techniques in binary analysis,” in Proc. IEEE Symposium on Security and Privacy (S&P) , 2016

2016

Earlier work this paper cites.

X. Xu, C. Liu, Q. Feng, H. Yin, L. Song, and D. Song, “Neural network-based graph embedding for cross-platform binary code similarity detection,” in Proc. ACM CCS , 2017

2017

Earlier work this paper cites.

Z. Li, D. Zou, S. Xu, X. Ou, H. Jin, S. Wang, Z. Deng, and Y. Zhong, “Vuldeepecker: A deep learning-based system for vulnerability detection,” 2018, arXiv:1801.01681

Original

2018

Earlier work this paper cites.

Y. Zhou, S. Liu, J. Siow, X. Du, and Y. Liu, “Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks,” in Proc. NeurIPS , 2019

2019

Earlier work this paper cites.

D. She, K. Pei, D. Epstein, J. Yang, B. Ray, and S. Jana, “Neuzz: Efficient fuzzing with neural program smoothing,” in Proc. IEEE Symposium on Security and Privacy (S&P) , 2019

2019

Earlier work this paper cites.

J. Lacomis, P. Yin, E. Schwartz, M. Allamanis, C. Le Goues, G. Neubig, and B. Vasilescu, “Dire: A neural approach to decompiled identifier naming,” in Proc. IEEE/ACM ASE , 2019

2019

Earlier work this paper cites.

A. Fioraldi, D. Maier, H. Eißfeldt, and M. Heuse, “AFL++ : Combining incremental steps of fuzzing research,” in Proc. USENIX WOOT 20 , 2020

2020

Earlier work this paper cites.

P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive nlp tasks,” in Proc. NeurIPS , 2020

2020

Earlier work this paper cites.

J. Fan, Y. Li, S. Wang, and T. N. Nguyen, “A c/c++ code vulnerability dataset with code changes and cve summaries,” in Proceedings of the 17th International Conference on Mining Software Repositories , ser. MSR ’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 508–512. [Online]. Available: https://doi.org/10.1145/3379597.3387501

2020

Earlier work this paper cites.

D. Wu, D. Gao, R. H. Deng, and C. Rocky K. C., “When program analysis meets bytecode search: Targeted and efficient inter-procedural analysis of modern android apps in backdroid,” in Proc. IEEE/IFIP DSN , 2021

2021

Earlier work this paper cites.

X. Li, Y. Qu, and H. Yin, “Palmtree: Learning an assembly language model for instruction embedding,” in Proc. ACM CCS , 2021

2021

Earlier work this paper cites.

J. Johnson, M. Douze, and H. Jégou, “Billion-scale similarity search with gpus,” IEEE Trans. Big Data , 2021

2021

Earlier work this paper cites.

S. Chakraborty, R. Krishna, Y. Ding, and B. Ray, “Deep learning based vulnerability detection: Are we there yet?” IEEE Trans. Softw. Eng. , 2022

2022

Earlier work this paper cites.

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions with human feedback,” 2022, arXiv:2203.02155

Original

2022

Earlier work this paper cites.

T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” in Proc. NeurIPS , 2022

2022

Earlier work this paper cites.

C. Thapa, S. I. Jang, M. E. Ahmed, S. Camtepe, J. Pieprzyk, and S. Nepal, “Transformer-based language models for software vulnerability detection,” in Proc. ACM ACSAC , 2022

2022

Earlier work this paper cites.

J. Wei, X. Wang, D. Schuurmans, M. Bosma, b. ichter, F. Xia, E. Chi, Q. V. Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” in Proc. NeurIPS , 2022

2022

Earlier work this paper cites.

N. Lambert, L. Castricato, L. von Werra, and A. Havrilla, “Illustrating reinforcement learning from human feedback (rlhf),” Hugging Face Blog , 2022, https://huggingface.co/blog/rlhf

2022

Earlier work this paper cites.

Q. Chen, J. Lacomis, E. J. Schwartz, C. L. Goues, G. Neubig, and B. Vasilescu, “Augmenting decompiler output with learned variable names and types,” in Proc. USENIX Security , 2022

2022

Earlier work this paper cites.

M. Alqarni and A. Azim, “Low level source code vulnerability detection using advanced bert language model,” in Proc. CANAI , 2022

2022

Earlier work this paper cites.

Y. Fang, D. Wu, X. Yi, S. Wang, Y. Chen, M. Chen, Y. Liu, and L. Jiang, “Beyond “protected” and “private”: An empirical security analysis of custom function modifiers in smart contracts,” in Proc. ACM ISSTA , 2023

2023

Earlier work this paper cites.

X. Yi, Y. Fang, D. Wu, and L. Jiang, “BlockScope: Detecting and Investigating Propagated Vulnerabilities in Forked Blockchain Projects,” in Proc. ISOC NDSS , 2023

2023

Earlier work this paper cites.

Y. Chen, Z. Ding, L. Alowain, X. Chen, and D. Wagner, “Diversevul: A new vulnerable source code dataset for deep learning based vulnerability detection,” in Proc. RAID , 2023

2023

Earlier work this paper cites.

Y. Deng, C. S. Xia, H. Peng, C. Yang, and L. Zhang, “Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,” in Proc. ACM ISSTA , 2023

2023

Earlier work this paper cites.

Y. Deng, C. S. Xia, C. Yang, S. D. Zhang, S. Yang, and L. Zhang, “Large language models are edge-case generators: Crafting unusual programs for fuzzing deep learning libraries,” in Proc. IEEE/ACM ICSE , 2023

2023

Earlier work this paper cites.

R. Meng, M. Mirchev, M. Bohme, and A. Roychoudhury, “Large language model guided protocol fuzzing,” in Proc. ISOC NDSS , 2023

2023

Cited alongside, same era.

S. Hu, T. Huang, F. Ilhan, S. Tekin, and L. Liu, “Large language model-powered smart contract vulnerability detection: New perspectives,” in Proc. IEEE TPS-ISA , 2023

2023

Cited alongside, same era.

H. Li, Y. Hao, Y. Zhai, and Z. Qian, “Enhancing static analysis for practical bug detection: An llm-integrated approach,” in Proc. ACM on Programming Languages (OOPSLA) , 2023

2023

Cited alongside, same era.

P. Liu, C. Sun, Y. Zheng, X. Feng, C. Qin, Y. Wang, Z. Li, and L. Sun, “Harnessing the power of llm to support binary taint analysis,” 2023, arXiv:2310.08275

Original

2023

Cited alongside, same era.

I. David, L. Zhou, K. Qin, D. Song, L. Cavallaro, and A. Gervais, “Do you still need a manual smart contract audit?” 2023, arXiv:2306.12338

C. S. Xia, M. Paltenghi, J. L. Tian, M. Pradel, and L. Zhang, “Fuzz4all: Universal fuzzing with large language models,” 2024, arXiv:2308.04748

Original

2024

Closest in time.

Y. Sun, D. Wu, Y. Xue, H. Liu, H. Wang, Z. Xu, X. Xie, and Y. Liu, “GPTScan: Detecting logic vulnerabilities in smart contracts by combining gpt with program analysis,” in Proc. IEEE/ACM ICSE , 2024

2024

Closest in time.

Y. Ding, Y. Fu, O. Ibrahim, C. Sitawarin, X. Chen, B. Alomair, D. Wagner, B. Ray, and Y. Chen, “Vulnerability detection with code language models: How far are we?” in 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE) , 2024, pp. 469–481

2024

Closest in time.

L. Zhang, K. Li, K. Sun, D. Wu, Y. Liu, H. Tian, and Y. Liu, “Acfix: Guiding llms with mined common rbac practices for context-aware repair of access control vulnerabilities in smart contracts,” 2024, arXiv:2403.06838

Original

2024

Closest in time.