Large language models (LLMs), as epitomized by models like ChatGPT, have revolutionized the field of natural language processing (NLP). Along with this trend, code-based large language models such as StarCoder, WizardCoder, and CodeLlama have emerged, trained extensively on vast repositories of code data. Yet, inherent in their design, these models primarily focus on generative tasks like code generation, code completion, and comment generation, and general support for multiple programming languages. While the generic abilities of code LLMs are useful for many programmers, the area of high-performance computing (HPC) has a narrower set of requirements that make a smaller and more domain-specific LM a smarter choice. This paper introduces OMPGPT, a novel model meticulously designed to harness the inherent strengths of language models for OpenMP pragma generation. Furthermore, we adopt and adapt prompt engineering techniques from the NLP domain to create chain-of-OMP, an innovative strategy designed to enhance OMPGPT's effectiveness. Our extensive evaluations demonstrate that OMPGPT outperforms existing large language models specialized in OpenMP tasks and maintains a notably smaller size, aligning it more closely with the typical hardware constraints of HPC environments. We consider our contribution as a pivotal bridge, connecting the advantage of language models with the specific demands of HPC tasks. The success of OMPGPT lays a solid foundation, suggesting its potential applicability and adaptability to a wider range of HPC tasks, thereby opening new avenues in the field of computational efficiency and effectiveness.

翻译：大型语言模型（LLMs），以ChatGPT等模型为代表，已彻底改变了自然语言处理（NLP）领域。伴随这一趋势，基于代码的大型语言模型（如StarCoder、WizardCoder和CodeLlama）相继涌现，这些模型在大量代码数据仓库上进行了广泛训练。然而，其设计本质决定了这些模型主要聚焦于生成式任务（例如代码生成、代码补全和注释生成），并提供对多种编程语言的通用支持。虽然代码LLMs的通用能力对许多程序员有益，但高性能计算（HPC）领域具有一组更窄的需求，这使得更小且更面向特定领域的语言模型成为更明智的选择。本文提出了OMPGPT，这是一种精心设计的新颖模型，旨在利用语言模型在OpenMP pragma生成中的固有优势。此外，我们借鉴并改编了NLP领域的提示工程技术，创建了chain-of-OMP——一种旨在增强OMPGPT有效性的创新策略。我们广泛的评估表明，OMPGPT在OpenMP任务上的表现优于现有的大型语言模型，同时保持了显著更小的模型规模，使其更契合HPC环境典型的硬件约束。我们认为我们的贡献是一座关键的桥梁，连接了语言模型的优势与HPC任务的特定需求。OMPGPT的成功奠定了坚实基础，表明其在更广泛的HPC任务中具有潜在的应用价值和适应性，从而为计算效率与效能领域开辟了新途径。