Parallelizing sequentially written programs is a challenging task. Even experienced developers need to spend considerable time finding parallelism opportunities and then actually writing parallel versions of sequentially written programs. To address this issue, we present AUTOPARLLM, a framework for automatically discovering parallelism and generating the parallel version of the sequentially written program. Our framework consists of two major components: i) a heterogeneous Graph Neural Network (GNN) based parallelism discovery and parallel pattern detection module, and ii) an LLM-based code generator to generate the parallel counterpart of the sequential programs. We use the GNN to learn the flow-aware characteristics of the programs to identify parallel regions in sequential programs and then construct an enhanced prompt using the GNN's results for the LLM-based generator to finally produce the parallel counterparts of the sequential programs. We evaluate AUTOPARLLM on 11 applications of 2 well-known benchmark suites: NAS Parallel Benchmark and Rodinia Benchmark. Our results show that AUTOPARLLM is indeed effective in improving the state-of-the-art LLM-based models for the task of parallel code generation in terms of multiple code generation metrics. AUTOPARLLM also improves the average runtime of the parallel code generated by the state-of-the-art LLMs by as high as 3.4% and 2.9% for the NAS Parallel Benchmark and Rodinia Benchmark respectively. Additionally, to overcome the issue that well-known metrics for translation evaluation have not been optimized to evaluate the quality of the generated parallel code, we propose OMPScore for evaluating the quality of the generated code. We show that OMPScore exhibits a better correlation with human judgment than existing metrics, measured by up to 75% improvement of Spearman correlation.

翻译：顺序编写程序的并行化是一项具有挑战性的任务。即使经验丰富的开发人员也需要花费大量时间发现并行化机会，并实际编写顺序程序的并行版本。为解决这一问题，我们提出AUTOPARLLM框架，该系统可自动发现并行性并生成顺序程序的并行版本。该框架包含两个核心组件：i) 基于异构图神经网络(GNN)的并行性发现和并行模式检测模块，以及ii) 基于大语言模型(LLM)的代码生成器，用于生成顺序程序的并行代码。我们利用GNN学习程序的流感知特征以识别顺序程序中的并行区域，随后将GNN分析结果构建为增强提示，引导基于LLM的生成器最终生成顺序程序的并行代码。我们在两个知名基准测试套件（NAS并行基准测试和Rodinia基准测试）的11个应用上评估了AUTOPARLLM。结果表明，AUTOPARLLM在多个代码生成指标上有效提升了现有最先进基于LLM模型在并行代码生成任务中的表现。对于NAS并行基准测试和Rodinia基准测试，AUTOPARLLM分别将最先进LLM生成的并行代码平均运行时间最高缩短了3.4%和2.9%。此外，针对现有翻译评估指标未能优化以评估生成并行代码质量的问题，我们提出OMPScore用于评估生成代码质量。实验表明，OMPScore与人类判断的相关性优于现有指标，斯皮尔曼相关系数提升高达75%。