Large language models (LLMs)such as ChatGPT have significantly advanced the field of Natural Language Processing (NLP). This trend led to the development of code-based large language models such as StarCoder, WizardCoder, and CodeLlama, which are trained extensively on vast repositories of code and programming languages. While the generic abilities of these code LLMs are useful for many programmers in tasks like code generation, the area of high-performance computing (HPC) has a narrower set of requirements that make a smaller and more domain-specific model a smarter choice. This paper presents OMPGPT, a novel domain-specific model meticulously designed to harness the inherent strengths of language models for OpenMP pragma generation. Furthermore, we leverage prompt engineering techniques from the NLP domain to create Chain-of-OMP, an innovative strategy designed to enhance OMPGPT's effectiveness. Our extensive evaluations demonstrate that OMPGPT outperforms existing large language models specialized in OpenMP tasks and maintains a notably smaller size, aligning it more closely with the typical hardware constraints of HPC environments. We consider our contribution as a pivotal bridge, connecting the advantage of language models with the specific demands of HPC tasks.

翻译：以ChatGPT为代表的大型语言模型显著推进了自然语言处理领域的发展。这一趋势催生了基于代码的大型语言模型（如StarCoder、WizardCoder和CodeLlama），这些模型在庞大的代码库与编程语言资源上进行了广泛训练。尽管这些代码通用语言模型在代码生成等任务中对许多程序员具有实用价值，但高性能计算领域的需求更为专精，使得更小规模、面向特定领域的模型成为更明智的选择。本文提出OMPGPT——一种精心设计的新型领域专用模型，旨在发挥语言模型在OpenMP编译指导语句生成中的固有优势。此外，我们借鉴自然语言处理领域的提示工程技巧，提出了Chain-of-OMP这一创新策略，以增强OMPGPT的有效性。广泛评估表明，OMPGPT在OpenMP任务上优于现有的大型语言模型，同时保持显著更小的模型规模，更贴合高性能计算环境的典型硬件约束。我们认为该贡献是一座连接语言模型优势与HPC任务特定需求的关键桥梁。