2023

Fixing Hardware Security Bugs with Large Language Models

Ahmad, Baleegh, Thakur, Shailja, Tan, Benjamin et al.

Understand

Novel AI-based code-writing Large Language Models (LLMs) such as OpenAI's Codex have demonstrated capabilities in many coding-adjacent domains.

In this work we consider how LLMs maybe leveraged to automatically repair security relevant bugs present in hardware designs.
We focus on bug repair in code written in the Hardware Description Language Verilog.
For this study we build a corpus of domain-representative hardware security bugs.

Built on

Hardware security in practice: Challenges and opportunities. In 2011 IEEE International Symposium on Hardware-Oriented Security and Trust . 93–98
Nachiketh Potlapally. 2011 · 2011
Earlier work this paper cites.
Automated feedback generation for introductory programming assignments. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’13) . Association for Computing Machinery, New York, NY, USA, 15–26
Rishabh Singh, Sumit Gulwani, and Armando Solar-Lezama. 2013 · 2013
Earlier work this paper cites.
A Primer on Hardware Security: Models, Methods, and Metrics
Masoud Rostami, Farinaz Koushanfar, and Ramesh Karri. 2014 · 2014
Earlier work this paper cites.
Automated Synthesis of Optimized Circuits for Secure Computation. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS ’15) . Association for Computing Machinery, New York, NY, USA, 1504–1517
Daniel Demmler, Ghada Dessouky, Farinaz Koushanfar, Ahmad-Reza Sadeghi, Thomas Schneider, and Shaza Zeitouni. 2015 · 2015
Earlier work this paper cites.
Register transfer level information flow tracking for provably secure hardware design. In Design, Automation Test in Europe Conference Exhibition (DATE), 2017 . 1691–1696
Armaiti Ardeshiricham, Wei Hu, Joshua Marxen, and Ryan Kastner. 2017 · 2017
Earlier work this paper cites.
VuRLE: Automatic Vulnerability Detection and Repair by Learning from Examples. In Computer Security – ESORICS 2017 (Lecture Notes in Computer Science) , Simon N. Foley, Dieter Gollmann, and Einar Snekkenes (Eds.). Springer International Publishing, Cham, 229–246
Siqi Ma, Ferdian Thung, David Lo, Cong Sun, and Robert H. Deng. 2017 · 2017
Earlier work this paper cites.
Meltdown: Reading Kernel Memory from User Space. 973–990
Moritz Lipp, Michael Schwarz, Daniel Gruss, Thomas Prescher, Werner Haas, Anders Fogh, Jann Horn, Stefan Mangard, Paul Kocher, Daniel Genkin, Yuval Yarom, and Mike Hamburg. 2018 · 2018
Earlier work this paper cites.
The Living Review on Automated Program Repair
Martin Monperrus. 2018 · 2018
Earlier work this paper cites.
Hardware | OpenTitan Documentation
2019 · 2019
Earlier work this paper cites.
VeriSketch: Synthesizing Secure Hardware Designs with Timing-Sensitive Information Flow Properties. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS ’19) . Association for Computing Machinery, New York, NY, USA, 1623–1638
Armaiti Ardeshiricham, Yoshiki Takashima, Sicun Gao, and Ryan Kastner. 2019 · 2019
Earlier work this paper cites.
Getafix: learning to fix bugs automatically
Johannes Bader, Andrew Scott, Michael Pradel, and Satish Chandra. 2019 · 2019
Earlier work this paper cites.
Hardfails: Insights into Software-Exploitable Hardware Bugs. In Proceedings of the 28th USENIX Conference on Security Symposium (SEC’19) . USENIX Association, Santa Clara, CA, USA, 213–230
Ghada Dessouky, David Gens, Patrick Haney, Garrett Persyn, Arun Kanuparthi, Hareesh Khattri, Jason Fung, Ahmad-Reza Sadeghi, and Jeyavijayan Rajendran. 2019 · 2019
Earlier work this paper cites.
Processor Hardware Security Vulnerabilities and their Detection by Unique Program Execution Checking. In 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE) . 994–999
Mohammad Rahmani Fadiheh, Dominik Stoffel, Clark Barrett, Subhasish Mitra, and Wolfgang Kunz. 2019 · 2019
Earlier work this paper cites.
Automated program repair
Claire Le Goues, Michael Pradel, and Abhik Roychoudhury. 2019 · 2019
Earlier work this paper cites.
Spectre Attacks: Exploiting Speculative Execution. In 2019 IEEE Symposium on Security and Privacy (SP) . 1–19
Paul Kocher, Jann Horn, Anders Fogh, Daniel Genkin, Daniel Gruss, Werner Haas, Mike Hamburg, Moritz Lipp, Stefan Mangard, Thomas Prescher, Michael Schwarz, and Yuval Yarom. 2019 · 2019
Earlier work this paper cites.
An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation
Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk. 2019 · 2019
Earlier work this paper cites.
LoopFix: an approach to automatic repair of buggy loops
Weichao Wang, Zhaopeng Meng, Zan Wang, Shuang Liu, and Jianye Hao. 2019 · 2019
Earlier work this paper cites.
Hunting Security Bugs in SoC Designs: Lessons Learned
Mohammad Mahdi Bidmeshki, Yunjie Zhang, Monir Zaman, Liwei Zhou, and Yiorgos Makris. 2021 · 2020
Earlier work this paper cites.

Similar

SCRIPT: A CAD Framework for Power Side-channel Vulnerability Assessment Using Information Flow Tracking and Pattern Generation
Adib Nahiyan, Jungmin Park, Miao He, Yousef Iskander, Farimah Farahmandi, Domenic Forte, and Mark Tehranipoor. 2020 · 2020
Cited alongside, same era.
DAVE: Deriving Automatically Verilog from English. In 2020 ACM/IEEE 2nd Workshop on Machine Learning for CAD (MLCAD) . 27–32
Hammond Pearce, Benjamin Tan, and Ramesh Karri. 2020 · 2020
Cited alongside, same era.
Special Session: CAD for Hardware Security - Automation is Key to Adoption of Solutions. In 2021 IEEE 39th VLSI Test Symposium (VTS) . 1–10
Sohrab Aftabjahani, Ryan Kastner, Mark Tehranipoor, Farimah Farahmandi, Jason Oberg, Anders Nordstrom, Nicole Fern, and Alric Althoff. 2021 · 2021
Cited alongside, same era.
TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer. In Proceedings of the 38th International Conference on Machine Learning . PMLR, 780–791
Berkay Berabi, Jingxuan He, Veselin Raychev, and Martin Vechev. 2021 · 2021
Cited alongside, same era.
Evaluating Large Language Models Trained on Code
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021 · 2021
Cited alongside, same era.
Generating bug-fixes using pretrained transformers. In Proceedings of the 5th ACM SIGPLAN International Symposium on Machine Programming (MAPS 2021) . Association for Computing Machinery, New York, NY, USA, 1–8
Dawn Drain, Chen Wu, Alexey Svyatkovskiy, and Neel Sundaresan. 2021 · 2021
Cited alongside, same era.
GitHub Copilot · Your AI pair programmer
GitHub. 2021 · 2021
Cited alongside, same era.
ReFixar: Multi-version Reasoning for Automated Repair of Regression Errors. In 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) . 162–172
Xuan-Bach D. Le and Quang Loc Le. 2021 · 2021
Cited alongside, same era.
FAPR: Fast and Accurate Program Repair for Introductory Programming Courses
Original
Yunlong Lu, Na Meng, and Wenxin Li. 2021 · 2021
Cited alongside, same era.
OpenAI Codex
OpenAI. 2021 · 2021
Cited alongside, same era.
Fuzzing Hardware Like Software
Original
Timothy Trippel, Kang G. Shin, Alex Chernyakhovsky, Garret Kelly, Dominic Rizzo, and Matthew Hicks. 2021 · 2021
Cited alongside, same era.
VarFix: balancing edit expressiveness and search effectiveness in automated program repair. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2021) . Association for Computing Machinery, New York, NY, USA, 354–366
Chu-Pan Wong, Priscila Santiesteban, Christian Kästner, and Claire Le Goues. 2021 · 2021
Cited alongside, same era.

Then

Sorald: Automatic Patch Suggestions for SonarQube Static Analysis Violations
Original
Khashayar Etemadi, Nicolas Harrand, Simon Larsen, Haris Adzemovic, Henry Luong Phu, Ashutosh Verma, Fernanda Madeiral, Douglas Wikstrom, and Martin Monperrus. 2022 · 2022
Later among the works it cites.
HACK@DAC21 – HacK@EVENT
HACK@EVENT. 2022 · 2022
Later among the works it cites.
Jasper Superlint App
jasperlint. 2022 · 2022
Later among the works it cites.
Design Guidelines for Prompt Engineering Text-to-Image Generative Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22) . Association for Computing Machinery, New York, NY, USA, 1–23
Vivian Liu and Lydia B Chilton. 2022 · 2022
Later among the works it cites.
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
Original
Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022 · 2022
Later among the works it cites.
A Taxonomy of Prompt Modifiers for Text-To-Image Generation
Original
Jonas Oppenlaender. 2022 · 2022
Later among the works it cites.
Automatically Mitigating Vulnerabilities in x86 Binary Programs via Partially Recompilable Decompilation
Original
Pemma Reiter, Hui Jun Tay, Westley Weimer, Adam Doupé, Ruoyu Wang, and Stephanie Forrest. 2022 · 2022
Later among the works it cites.
Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models
Hendrik Strobelt, Albert Webson, Victor Sanh, Benjamin Hoover, Johanna Beyer, Hanspeter Pfister, and Alexander M. Rush. 2023 · 2022
Later among the works it cites.
Benchmarking Large Language Models for Automated Verilog RTL Code Generation
Original
Shailja Thakur, Baleegh Ahmad, Zhenxing Fan, Hammond Pearce, Benjamin Tan, Ramesh Karri, Brendan Dolan-Gavitt, and Siddharth Garg. 2022 · 2022
Later among the works it cites.
TheHuzz: Instruction Fuzzing of Processors Using Golden-Reference Models for Finding Software-Exploitable Vulnerabilities
Original
Aakash Tyagi, Addison Crump, Ahmad-Reza Sadeghi, Garrett Persyn, Jeyavijayan Rajendran, Patrick Jauernig, and Rahul Kande. 2022 · 2022
Later among the works it cites.
Synopsys VC SpyGlass Lint
vclint. 2022 · 2022
Later among the works it cites.
Fault Localization for Hardware Design Code with Time-Aware Program Spectrum. In 2022 IEEE 40th International Conference on Computer Design (ICCD) . 537–544
Jiang Wu, Zhuo Zhang, Deheng Yang, Xiankai Meng, Jiayu He, Xiaoguang Mao, and Yan Lei. 2022 · 2022
Later among the works it cites.
Example-Based Vulnerability Detection and Repair in Java Code
Original
Ying Zhang, Ya Xiao, Md Mahir Asef Kabir, Danfeng, Yao, and Na Meng. 2022 · 2022
Later among the works it cites.
Large Language Models Are Human-Level Prompt Engineers
Original
Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022 · 2022
Later among the works it cites.
Trust-Hub.org
2023 · 2023
Closest in time.
Examining Zero-Shot Vulnerability Repair with Large Language Models. IEEE Computer Society, 1–18
Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. 2022 · 2023
Closest in time.
Artifacts for “Large Language Models Can Fix Hardware Security Bugs”
Anonymized for review. 2023 · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…