2024

LLM-Aided Testbench Generation and Bug Detection for Finite-State Machines

Bhandari, Jitendra, Knechtel, Johann, Narayanaswamy, Ramesh et al.

Understand

This work investigates the potential of tailoring Large Language Models (LLMs), specifically GPT3.5 and GPT4, for the domain of chip testing.

A key aspect of chip design is functional testing, which relies on testbenches to evaluate the functionality and coverage of Register-Transfer Level (RTL) designs.
We aim to enhance testbench generation by incorporating feedback from commercial-grade Electronic Design Automation (EDA) tools into LLMs.
Through iterative feedback from these tools, we refine the testbenches to achieve improved test coverage.

J. Takahashi et al. , “Testing for high assurance system by fsm,” IEICE TRANSACTIONS on Information and Systems , vol. 86, no. 10, pp. 2114–2120, 2003
2003
Earlier work this paper cites.
OpenAI, “GPT-4,” Mar. 2023. [Online]. Available: https://openai.com/research/gpt-4
2023
Earlier work this paper cites.
M. Liu et al. , “Verilogeval: Evaluating large language models for verilog code generation,” in 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD) . IEEE, 2023, pp. 1–8
2023
Earlier work this paper cites.
S. Thakur et al. , “Autochip: Automating hdl generation using llm feedback,” arXiv preprint arXiv:2311.04887 , 2023
Original
2023
Earlier work this paper cites.
S. Thakur et al. , “Verigen: A large language model for verilog code generation,” ACM Transactions on Design Automation of Electronic Systems , 2023
2023
Earlier work this paper cites.
R. Kande et al. , “Llm-assisted generation of hardware assertions,” arXiv preprint arXiv:2306.14027 , 2023
Original
2023
Earlier work this paper cites.

M. Liu et al. , “Chipnemo: Domain-adapted llms for chip design,” arXiv preprint arXiv:2311.00176 , 2023
Original
2023
Cited alongside, same era.
R. Zhong et al. , “Llm4eda: Emerging progress in large language models for electronic design automation,” arXiv preprint arXiv:2401.12224 , 2023
Original
2023
Cited alongside, same era.
J. Blocklove et al. , “Chip-chat: Challenges and opportunities in conversational hardware design,” in 2023 ACM/IEEE 5th Workshop on Machine Learning for CAD (MLCAD) . IEEE, 2023, pp. 1–6
2023
Cited alongside, same era.
Y. Fu et al. , “Gpt4aigchip: Towards next-generation ai accelerator design automation via large language models,” in 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD) . IEEE, 2023, pp. 1–9
2023
Cited alongside, same era.
K. Chang et al. , “Chipgpt: How far are we from natural language hardware design,” arXiv preprint arXiv:2305.14019 , 2023
Original
2023
Cited alongside, same era.

“Problem sets - HDLBits,” 2023. [Online]. Available: https://hdlbits.01xz.net/wiki/Problem_sets
2023
Later among the works it cites.
Y. Lu et al. , “Rtllm: An open-source benchmark for design rtl generation with large language model,” in 2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC) . IEEE, 2024, pp. 722–727
2024
Closest in time.
W. Fang et al. , “Assertllm: Generating and evaluating hardware verification assertions from design specifications via multi-llms,” arXiv preprint arXiv:2402.00386 , 2024
Original
2024
Closest in time.
H. Wu et al. , “Chateda: A large language model powered autonomous agent for eda,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2024
2024
Closest in time.
J. Bhandari. (2024) LLM-Aided-Testbench-Generation-for-FSM. [Online]. Available: https://github.com/jitendra-bhandari/LLM-Aided-Testbench-Generation-for-FSM
2024
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…