In this study, we explore the capability of Large Language Models (LLMs) to automate hardware design by generating high-quality Verilog code, a common language for designing and modeling digital systems. We fine-tune pre-existing LLMs on Verilog datasets compiled from GitHub and Verilog textbooks. We evaluate the functional correctness of the generated Verilog code using a specially designed test suite, featuring a custom problem set and testing benches. Here, our fine-tuned open-source CodeGen-16B model outperforms the commercial state-of-the-art GPT-3.5-turbo model with a 1.1% overall increase. Upon testing with a more diverse and complex problem set, we find that the fine-tuned model shows competitive performance against state-of-the-art gpt-3.5-turbo, excelling in certain scenarios. Notably, it demonstrates a 41% improvement in generating syntactically correct Verilog code across various problem categories compared to its pre-trained counterpart, highlighting the potential of smaller, in-house LLMs in hardware design automation.

翻译：本研究探索了大型语言模型（LLMs）通过生成高质量Verilog代码来自动化硬件设计的能力，Verilog是数字系统设计与建模的常用语言。我们在从GitHub和Verilog教科书中编译的Verilog数据集上对预训练的LLMs进行了微调。我们使用专门设计的测试套件（包含自定义问题集和测试平台）评估生成的Verilog代码的功能正确性。在此框架下，我们微调的开源CodeGen-16B模型以1.1%的总提升幅度超越了商业上最先进的GPT-3.5-turbo模型。通过更具多样性和复杂性的问题集测试，我们发现微调模型与最先进的gpt-3.5-turbo相比展现出竞争性表现，在某些场景中表现优异。值得注意的是，与预训练版本相比，该模型在不同问题类别中生成语法正确Verilog代码的能力提升了41%，凸显了小型内部LLMs在硬件设计自动化中的潜力。