ChipNeMo aims to explore the applications of large language models (LLMs) for industrial chip design. Instead of directly deploying off-the-shelf commercial or open-source LLMs, we instead adopt the following domain adaptation techniques: custom tokenizers, domain-adaptive continued pretraining, supervised fine-tuning (SFT) with domain-specific instructions, and domain-adapted retrieval models. We evaluate these methods on three selected LLM applications for chip design: an engineering assistant chatbot, EDA script generation, and bug summarization and analysis. Our results show that these domain adaptation techniques enable significant LLM performance improvements over general-purpose base models across the three evaluated applications, enabling up to 5x model size reduction with similar or better performance on a range of design tasks. Our findings also indicate that there's still room for improvement between our current results and ideal outcomes. We believe that further investigation of domain-adapted LLM approaches will help close this gap in the future.

翻译：ChipNeMo旨在探索大语言模型在工业芯片设计中的应用。我们并未直接部署现成的商用或开源大语言模型，而是采用以下领域自适应技术：自定义分词器、领域自适应持续预训练、基于领域特定指令的监督微调（SFT）以及领域自适应检索模型。我们在芯片设计的三个精选大语言模型应用场景中评估了这些方法：工程助手聊天机器人、EDA脚本生成以及错误摘要与分析。结果表明，上述领域自适应技术使大语言模型在三个应用场景中的性能显著优于通用基础模型，并在多种设计任务中以相似或更优的性能实现了高达5倍的模型体积缩减。研究同时指出，当前结果与理想效果之间仍存在改进空间。我们相信，对领域自适应大语言模型方法的进一步探索将有助于在未来弥合这一差距。