Parallelizing sequentially written programs is a challenging task. Even experienced developers need to spend considerable time finding parallelism opportunities and then actually writing parallel versions of sequentially written programs. To address this issue, we present AUTOPARLLM, a framework for automatically discovering parallelism and generating the parallel version of the sequentially written program. Our framework consists of two major components: i) a heterogeneous Graph Neural Network (GNN) based parallelism discovery and parallel pattern detection module, and ii) an LLM-based code generator to generate the parallel counterpart of the sequential programs. We use the GNN to learn the flow-aware characteristics of the programs to identify parallel regions in sequential programs and then construct an enhanced prompt using the GNN's results for the LLM-based generator to finally produce the parallel counterparts of the sequential programs. We evaluate AUTOPARLLM on 11 applications of 2 well-known benchmark suites: NAS Parallel Benchmark and Rodinia Benchmark. Our results show that AUTOPARLLM is indeed effective in improving the state-of-the-art LLM-based models for the task of parallel code generation in terms of multiple code generation metrics. AUTOPARLLM also improves the average runtime of the parallel code generated by the state-of-the-art LLMs by as high as 3.4% and 2.9% for the NAS Parallel Benchmark and Rodinia Benchmark respectively. Additionally, to overcome the issue that well-known metrics for translation evaluation have not been optimized to evaluate the quality of the generated parallel code, we propose OMPScore for evaluating the quality of the generated code. We show that OMPScore exhibits a better correlation with human judgment than existing metrics, measured by up to 75% improvement of Spearman correlation.

翻译：将顺序编写的程序进行并行化是一项具有挑战性的任务。即便是经验丰富的开发者也需要花费大量时间寻找并行化机会，并实际编写顺序程序的并行版本。为解决这一问题，我们提出了AUTOPARLLM框架，该框架能够自动发现并行性并生成顺序程序的并行版本。我们的框架包含两个主要组件：i) 基于异构图神经网络（GNN）的并行性发现与并行模式检测模块，以及ii) 基于大语言模型（LLM）的代码生成器，用于生成顺序程序的并行对应代码。我们利用GNN学习程序的流感知特征以识别顺序程序中的并行区域，然后基于GNN的结果构建增强提示，引导基于LLM的生成器最终生成顺序程序的并行版本。我们在两个知名基准测试套件（NAS并行基准测试和Rodinia基准测试）的11个应用上评估了AUTOPARLLM。结果表明，AUTOPARLLM在多项代码生成指标上有效提升了现有最优基于LLM模型的并行代码生成性能。对于NAS并行基准测试和Rodinia基准测试，AUTOPARLLM还将现有最优LLM生成的并行代码的平均运行时间分别提高了高达3.4%和2.9%。此外，针对现有翻译评估指标未针对生成并行代码质量进行优化的问题，我们提出了OMPScore用于评估生成代码的质量。实验表明，OMPScore与人类判断的相关性优于现有指标，其Spearman相关性提高了高达75%。