Automatic source-to-source parallelization of serial code for shared and distributed memory systems is a challenging task in high-performance computing. While many attempts were made to translate serial code into parallel code for a shared memory environment (usually using OpenMP), none has managed to do so for a distributed memory environment. In this paper, we propose a novel approach, called MPI-rical, for automated MPI code generation using a transformer-based model trained on approximately 25,000 serial code snippets and their corresponding parallelized MPI code out of more than 50,000 code snippets in our corpus (MPICodeCorpus). To evaluate the performance of the model, we first break down the serial code to MPI-based parallel code translation problem into two sub-problems and develop two research objectives: code completion defined as given a location in the source code, predict the MPI function for that location, and code translation defined as predicting an MPI function as well as its location in the source code. We evaluate MPI-rical on MPICodeCorpus dataset and on real-world scientific code benchmarks and compare its performance between the code completion and translation tasks. Our experimental results show that while MPI-rical performs better on the code completion task than the code translation task, the latter is better suited for real-world programming assistance, in which the tool suggests the need for an MPI function regardless of prior knowledge. Overall, our approach represents a significant step forward in automating the parallelization of serial code for distributed memory systems, which can save valuable time and resources for software developers and researchers. The source code used in this work, as well as other relevant sources, are available at: https://github.com/Scientific-Computing-Lab-NRCN/MPI-rical

翻译：共享内存与分布式内存系统上串行代码的自动源到源并行化是高性能计算中的一项挑战性任务。尽管已有许多尝试将串行代码转换为共享内存环境下的并行代码（通常使用OpenMP），但尚未有方法成功实现分布式内存环境的自动并行化。本文提出一种名为MPI-rical的新方法，利用基于Transformer的模型实现MPI代码自动生成。该模型在我们的语料库（MPICodeCorpus）中超过50,000个代码片段中，基于约25,000个串行代码片段及其对应的并行化MPI代码进行训练。为评估模型性能，我们首先将串行代码到MPI并行代码的翻译问题分解为两个子问题，并设定两个研究目标：代码补全（给定源代码中的位置，预测该位置的MPI函数）和代码翻译（同时预测MPI函数及其在源代码中的位置）。我们在MPICodeCorpus数据集和真实科学代码基准上评估MPI-rical，并比较其在代码补全与翻译任务上的性能。实验结果表明，尽管MPI-rical在代码补全任务上表现优于代码翻译任务，但后者更适用于实际编程辅助场景——工具可在无需先验知识的情况下主动建议需要插入的MPI函数。总体而言，我们的方法在自动化分布式内存系统串行代码并行化方面迈出了重要一步，可为软件开发者和研究人员节省宝贵的时间与资源。本工作使用的源代码及相关资源见：https://github.com/Scientific-Computing-Lab-NRCN/MPI-rical