ChipNeMo aims to explore the applications of large language models (LLMs) for industrial chip design. Instead of directly deploying off-the-shelf commercial or open-source LLMs, we instead adopt the following domain adaptation techniques: custom tokenizers, domain-adaptive continued pretraining, supervised fine-tuning (SFT) with domain-specific instructions, and domain-adapted retrieval models. We evaluate these methods on three selected LLM applications for chip design: an engineering assistant chatbot, EDA script generation, and bug summarization and analysis. Our results show that these domain adaptation techniques enable significant LLM performance improvements over general-purpose base models across the three evaluated applications, enabling up to 5x model size reduction with similar or better performance on a range of design tasks. Our findings also indicate that there's still room for improvement between our current results and ideal outcomes. We believe that further investigation of domain-adapted LLM approaches will help close this gap in the future.

翻译：ChipNeMo旨在探索大语言模型（LLM）在工业芯片设计中的应用。我们没有直接部署现成的商业或开源大语言模型，而是采用了以下领域自适应技术：定制分词器、领域自适应持续预训练、基于领域特定指令的监督微调（SFT），以及领域自适应检索模型。我们在芯片设计的三个选定大语言模型应用场景中评估了这些方法：工程辅助聊天机器人、EDA脚本生成，以及错误总结与分析。我们的结果表明，这些领域自适应技术使得大语言模型在三个评估应用中的性能显著优于通用基础模型，从而在多种设计任务中实现了最多5倍的模型尺寸缩减，同时性能相似或更优。我们的发现也表明，当前结果与理想结果之间仍存在改进空间。我们相信，对领域自适应大语言模型方法的进一步探究将在未来帮助缩小这一差距。