Optimizing scientific software is a difficult task because codebases are often large and complex, and performance can depend upon several factors including the algorithm, its implementation, and hardware among others. Causes of poor performance can originate from disparate sources and be difficult to diagnose. Recent years have seen a multitude of work that use large language models (LLMs) to assist in software development tasks. However, these tools are trained to model the distribution of code as text, and are not specifically designed to understand performance aspects of code. In this work, we introduce a reinforcement learning based methodology to align the outputs of code LLMs with performance. This allows us to build upon the current code modeling capabilities of LLMs and extend them to generate better performing code. We demonstrate that our fine-tuned model improves the expected speedup of generated code over base models for a set of benchmark tasks from 0.9 to 1.6 for serial code and 1.9 to 4.5 for OpenMP code.

翻译：优化科学软件是一项艰巨的任务，因为代码库通常庞大而复杂，性能可能取决于多种因素，包括算法、实现方式和硬件等。性能不佳的原因可能来自不同来源，且难以诊断。近年来，大量工作利用大语言模型（LLM）辅助软件开发任务。然而，这些工具被训练为将代码建模为文本分布，并未专门设计用于理解代码的性能方面。在本研究中，我们引入了一种基于强化学习的方法论，使代码LLM的输出与性能对齐。这使我们能够基于LLM现有的代码建模能力，将其扩展为生成性能更优的代码。我们证明，经过微调的模型在基准任务集上生成的代码相对于基础模型，在串行代码上将预期加速比从0.9提升至1.6，在OpenMP代码上从1.9提升至4.5。