Built on

P. Coussy and A. Morawiec, High-level synthesis . Springer, 2010, vol. 1
2010
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is All you Need,” in Advances in Neural Information Processing Systems , vol. 30. Curran Associates, Inc., 2017. [Online]. Available: https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
2017
Earlier work this paper cites.
H. Foster, “Wilson Research Group Functional Verification Study 2020,” Nov. 2020, section: News. [Online]. Available: https://blogs.sw.siemens.com/verificationhorizons/2020/11/05/part-1-the-2020-wilson-research-group-functional-verification-study/
2020
Earlier work this paper cites.
H. Pearce, B. Tan, and R. Karri, “DAVE: Deriving Automatically Verilog from English,” in 2020 ACM/IEEE 2nd Workshop on Machine Learning for CAD (MLCAD) , Nov. 2020, pp. 27–32
2020
Earlier work this paper cites.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language Models are Few-Shot Learners,” in Advances in Neural Information Processing Systems , H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 1877–1901. [Online]. Available: https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
2020
Earlier work this paper cites.
“OpenLane,” May 2023, original-date: 2020-07-20T19:35:02Z. [Online]. Available: https://github.com/The-OpenROAD-Project/OpenLane
2020
Earlier work this paper cites.
V. Hamolia and V. Melnyk, “A Survey of Machine Learning Methods and Applications in Electronic Design Automation,” in 2021 11th International Conference on Advanced Computer Information Technologies (ACIT) , Sep. 2021, pp. 757–760. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9548117
2021
Earlier work this paper cites.
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, “Evaluating Large Language Models Trained on Code,” Jul. 2021, arXiv:2107.03374 [cs]. [Online]. Available: http://arxiv.org/abs/2107.03374
Original
2021
Earlier work this paper cites.

Similar

OpenAI, “ChatGPT: Optimizing Language Models for Dialogue,” Nov. 2022. [Online]. Available: https://openai.com/blog/chatgpt/
2022
Cited alongside, same era.
——, “Introducing ChatGPT,” Nov. 2022. [Online]. Available: https://openai.com/blog/chatgpt
2022
Cited alongside, same era.
S. Thakur, B. Ahmad, Z. Fan, H. Pearce, B. Tan, R. Karri, B. Dolan-Gavitt, and S. Garg, “Benchmarking Large Language Models for Automated Verilog RTL Code Generation,” in 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE) , Apr. 2023, pp. 1–6, iSSN: 1558-1101. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10137086
2023
Cited alongside, same era.
J. Blocklove, S. Garg, R. Karri, and H. Pearce, “Chip-Chat: Challenges and Opportunities in Conversational Hardware Design,” in 2023 ACM/IEEE 5th Workshop on Machine Learning for CAD (MLCAD) , Sep. 2023, pp. 1–6. [Online]. Available: https://ieeexplore.ieee.org/document/10299874
2023
Cited alongside, same era.

Then

HuggingFace, “HuggingChat,” May 2023. [Online]. Available: https://huggingface.co/chat
2023
Later among the works it cites.
M. Liu, T.-D. Ene, R. Kirby, C. Cheng, N. Pinckney, R. Liang, J. Alben, H. Anand, S. Banerjee, I. Bayraktaroglu, B. Bhaskaran, B. Catanzaro, A. Chaudhuri, S. Clay, B. Dally, L. Dang, P. Deshpande, S. Dhodhi, S. Halepete, E. Hill, J. Hu, S. Jain, B. Khailany, K. Kunal, X. Li, H. Liu, S. Oberman, S. Omar, S. Pratty, J. Raiman, A. Sarkar, Z. Shao, H. Sun, P. P. Suthar, V. Tej, K. Xu, and H. Ren, “ChipNeMo: Domain-Adapted LLMs for Chip Design,” Nov. 2023, arXiv:2311.00176 [cs]. [Online]. Available: http://arxiv.org/abs/2311.00176
Original
2023
Later among the works it cites.
B. Ahmad, B. Tan, R. Karri, and H. Pearce, “FLAG: Finding Line Anomalies (in code) with Generative AI,” Jun. 2023, arXiv:2306.12643 [cs]. [Online]. Available: http://arxiv.org/abs/2306.12643
Original
2023
Later among the works it cites.
OpenAI, “GPT-4,” Mar. 2023. [Online]. Available: https://openai.com/research/gpt-4
2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Evaluating LLMs for Hardware Design and Test

Blocklove, Jason, Garg, Siddharth, Karri, Ramesh et al.

Understand

Large Language Models (LLMs) have demonstrated capabilities for producing code in Hardware Description Languages (HDLs).

However, most of the focus remains on their abilities to write functional code, not test code.
The hardware design process consists of both design and test, and so eschewing validation and verification leaves considerable potential benefit unexplored, given that a design and test framework may allow for progress towards full automation of the digital design pipeline.
In this work, we perform one of the first studies exploring how a LLM can both design and test hardware modules from provided specifications.

Built on

P. Coussy and A. Morawiec, High-level synthesis . Springer, 2010, vol. 1
2010
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is All you Need,” in Advances in Neural Information Processing Systems , vol. 30. Curran Associates, Inc., 2017. [Online]. Available: https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
2017
Earlier work this paper cites.
H. Foster, “Wilson Research Group Functional Verification Study 2020,” Nov. 2020, section: News. [Online]. Available: https://blogs.sw.siemens.com/verificationhorizons/2020/11/05/part-1-the-2020-wilson-research-group-functional-verification-study/
2020
Earlier work this paper cites.
H. Pearce, B. Tan, and R. Karri, “DAVE: Deriving Automatically Verilog from English,” in 2020 ACM/IEEE 2nd Workshop on Machine Learning for CAD (MLCAD) , Nov. 2020, pp. 27–32
2020
Earlier work this paper cites.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language Models are Few-Shot Learners,” in Advances in Neural Information Processing Systems , H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 1877–1901. [Online]. Available: https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
2020
Earlier work this paper cites.
“OpenLane,” May 2023, original-date: 2020-07-20T19:35:02Z. [Online]. Available: https://github.com/The-OpenROAD-Project/OpenLane
2020
Earlier work this paper cites.
V. Hamolia and V. Melnyk, “A Survey of Machine Learning Methods and Applications in Electronic Design Automation,” in 2021 11th International Conference on Advanced Computer Information Technologies (ACIT) , Sep. 2021, pp. 757–760. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9548117
2021
Earlier work this paper cites.
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, “Evaluating Large Language Models Trained on Code,” Jul. 2021, arXiv:2107.03374 [cs]. [Online]. Available: http://arxiv.org/abs/2107.03374
Original
2021
Earlier work this paper cites.

Similar

OpenAI, “ChatGPT: Optimizing Language Models for Dialogue,” Nov. 2022. [Online]. Available: https://openai.com/blog/chatgpt/
2022
Cited alongside, same era.
——, “Introducing ChatGPT,” Nov. 2022. [Online]. Available: https://openai.com/blog/chatgpt
2022
Cited alongside, same era.
S. Thakur, B. Ahmad, Z. Fan, H. Pearce, B. Tan, R. Karri, B. Dolan-Gavitt, and S. Garg, “Benchmarking Large Language Models for Automated Verilog RTL Code Generation,” in 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE) , Apr. 2023, pp. 1–6, iSSN: 1558-1101. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10137086
2023
Cited alongside, same era.
J. Blocklove, S. Garg, R. Karri, and H. Pearce, “Chip-Chat: Challenges and Opportunities in Conversational Hardware Design,” in 2023 ACM/IEEE 5th Workshop on Machine Learning for CAD (MLCAD) , Sep. 2023, pp. 1–6. [Online]. Available: https://ieeexplore.ieee.org/document/10299874
2023
Cited alongside, same era.

Then

HuggingFace, “HuggingChat,” May 2023. [Online]. Available: https://huggingface.co/chat
2023
Later among the works it cites.
M. Liu, T.-D. Ene, R. Kirby, C. Cheng, N. Pinckney, R. Liang, J. Alben, H. Anand, S. Banerjee, I. Bayraktaroglu, B. Bhaskaran, B. Catanzaro, A. Chaudhuri, S. Clay, B. Dally, L. Dang, P. Deshpande, S. Dhodhi, S. Halepete, E. Hill, J. Hu, S. Jain, B. Khailany, K. Kunal, X. Li, H. Liu, S. Oberman, S. Omar, S. Pratty, J. Raiman, A. Sarkar, Z. Shao, H. Sun, P. P. Suthar, V. Tej, K. Xu, and H. Ren, “ChipNeMo: Domain-Adapted LLMs for Chip Design,” Nov. 2023, arXiv:2311.00176 [cs]. [Online]. Available: http://arxiv.org/abs/2311.00176
Original
2023
Later among the works it cites.
B. Ahmad, B. Tan, R. Karri, and H. Pearce, “FLAG: Finding Line Anomalies (in code) with Generative AI,” Jun. 2023, arXiv:2306.12643 [cs]. [Online]. Available: http://arxiv.org/abs/2306.12643
Original
2023
Later among the works it cites.
OpenAI, “GPT-4,” Mar. 2023. [Online]. Available: https://openai.com/research/gpt-4
2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

M. Liu, N. Pinckney, B. Khailany, and H. Ren, “VerilogEval: Evaluating Large Language Models for Verilog Code Generation,” Sep. 2023, arXiv:2309.07544 [cs]. [Online]. Available: http://arxiv.org/abs/2309.07544

Original

2023

Cited alongside, same era.

“Tiny Tapeout,” May 2023. [Online]. Available: https://tinytapeout.com/

2023

Cited alongside, same era.

S. Pichai, “An important next step on our AI journey,” Feb. 2023. [Online]. Available: https://blog.google/technology/ai/bard-google-ai-search-updates/

2023

Cited alongside, same era.

“Welcome to RapidGPT | PrimisAI.” [Online]. Available: https://primis.ai/docs

Cited in the paper.

“Cadence JedAI Generative AI Solution for Chip, System, and Product Design.” [Online]. Available: https://www.cadence.com/en_US/home/solutions/joint-enterprise-data-ai-platform.html

Cited in the paper.

“Redefining Chip Design with AI-Powered EDA Tools | Synopsys.ai | Synopsys Blog.” [Online]. Available: https://www.synopsys.com/blogs/chip-design/synopsys-ai-eda-tools.html

Cited in the paper.

“Problem sets - HDLBits.” [Online]. Available: https://hdlbits.01xz.net/wiki/Problem_sets

Cited in the paper.

“Icarus Verilog.” [Online]. Available: http://iverilog.icarus.com/home

Cited in the paper.

S. Thakur, B. Ahmad, H. Pearce, B. Tan, B. Dolan-Gavitt, R. Karri, and S. Garg, “VeriGen: A Large Language Model for Verilog Code Generation,” ACM Transactions on Design Automation of Electronic Systems , Feb. 2024, just Accepted. [Online]. Available: https://dl.acm.org/doi/10.1145/3643681

2024

Closest in time.

A. f. b. review, “Repository for ”Evaluating LLMs for Hardware Design and Test”,” Apr. 2024. [Online]. Available: https://zenodo.org/records/10947127

2024

Closest in time.

R. Kande, H. Pearce, B. Tan, B. Dolan-Gavitt, S. Thakur, R. Karri, and J. Rajendran, “(Security) Assertions by Large Language Models,” IEEE Transactions on Information Forensics and Security , pp. 1–1, 2024, conference Name: IEEE Transactions on Information Forensics and Security. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10458667

2024

Closest in time.