There is an ever-present need for shared memory parallelization schemes to exploit the full potential of multi-core architectures. The most common parallelization API addressing this need today is OpenMP. Nevertheless, writing parallel code manually is complex and effort-intensive. Thus, many deterministic source-to-source (S2S) compilers have emerged, intending to automate the process of translating serial to parallel code. However, recent studies have shown that these compilers are impractical in many scenarios. In this work, we combine the latest advancements in the field of AI and natural language processing (NLP) with the vast amount of open-source code to address the problem of automatic parallelization. Specifically, we propose a novel approach, called OMPify, to detect and predict the OpenMP pragmas and shared-memory attributes in parallel code, given its serial version. OMPify is based on a Transformer-based model that leverages a graph-based representation of source code that exploits the inherent structure of code. We evaluated our tool by predicting the parallelization pragmas and attributes of a large corpus of (over 54,000) snippets of serial code written in C and C++ languages (Open-OMP-Plus). Our results demonstrate that OMPify outperforms existing approaches, the general-purposed and popular ChatGPT and targeted PragFormer models, in terms of F1 score and accuracy. Specifically, OMPify achieves up to 90% accuracy on commonly-used OpenMP benchmark tests such as NAS, SPEC, and PolyBench. Additionally, we performed an ablation study to assess the impact of different model components and present interesting insights derived from the study. Lastly, we also explored the potential of using data augmentation and curriculum learning techniques to improve the model's robustness and generalization capabilities.

翻译：多核架构潜能的充分释放始终离不开共享内存并行化方案的支持，而当今最主流的并行化API当属OpenMP。然而，手动编写并行代码不仅复杂且耗费大量人力。为此，业界涌现出诸多确定性源到源（S2S）编译器，旨在实现串行至并行代码的自动化转换。但近年研究表明，此类编译器在众多场景中缺乏实用性。本研究将人工智能与自然语言处理（NLP）领域最新进展与海量开源代码相结合，以攻克自动并行化这一难题。具体而言，我们提出名为OMPify的创新方法，能够针对给定串行代码版本，检测并预测其并行代码中应包含的OpenMP伪指令及共享内存属性。OMPify采用基于Transformer的模型架构，通过图结构表示源代码，充分挖掘代码的固有结构特征。我们通过预测C/C++语言编写的54,000余段串行代码片段（Open-OMP-Plus语料库）的并行化伪指令及属性来评估该工具性能。实验结果表明，OMPify在F1分数与准确率方面均显著优于现有方法（含通用型热门模型ChatGPT及专用模型PragFormer）。具体而言，OMPify在NAS、SPEC、PolyBench等常用OpenMP基准测试中达到90%准确率。此外，我们通过消融研究评估了模型各组件的贡献度，并从中提炼出若干有价值的发现。最后，我们还探索了运用数据增强与课程学习技术提升模型鲁棒性与泛化能力的潜力。