Growing heterogeneity and configurability in HPC architectures has made auto-tuning applications and runtime parameters on these systems very complex. Users are presented with a multitude of options to configure parameters. In addition to application specific solutions, a common approach is to use general purpose search strategies, which often might not identify the best configurations or their time to convergence is a significant barrier. There is, thus, a need for a general purpose and efficient tuning approach that can be easily scaled and adapted to various tuning tasks. We propose a technique for tuning parallel code regions that is general enough to be adapted to multiple tasks. In this paper, we analyze IR-based programming models to make task-specific performance optimizations. To this end, we propose the Multimodal Graph Neural Network and Autoencoder (MGA) tuner, a multimodal deep learning based approach that adapts Heterogeneous Graph Neural Networks and Denoizing Autoencoders for modeling IR-based code representations that serve as separate modalities. This approach is used as part of our pipeline to model a syntax, semantics, and structure-aware IR-based code representation for tuning parallel code regions/kernels. We extensively experiment on OpenMP and OpenCL code regions/kernels obtained from PolyBench, Rodinia, STREAM, DataRaceBench, AMD SDK, NPB, NVIDIA SDK, Parboil, SHOC, and LULESH benchmarks. We apply our multimodal learning techniques to the tasks of i) optimizing the number of threads, scheduling policy and chunk size in OpenMP loops and, ii) identifying the best device for heterogeneous device mapping of OpenCL kernels. Our experiments show that this multimodal learning based approach outperforms the state-of-the-art in all experiments.

翻译：高性能计算架构日益增长的异构性与可配置性使得自动调优应用及运行时参数变得极为复杂。用户面对大量参数配置选项。除特定应用解决方案外，通用搜索策略虽属常见方法，却往往无法确定最优配置，或收敛时间成为显著障碍。因此，亟需一种可轻松扩展并适配各类调优任务的通用高效调优方案。本文提出一种针对并行代码区域的调优技术，其通用性足以适配多种任务。我们通过分析基于中间表示的编程模型实现任务特定的性能优化。为此，提出多模态图神经网络与自编码器调优器（MGA）——一种基于多模态深度学习的方法，通过适配异构图神经网络与去噪自编码器对作为独立模态的基于中间表示的代码建模。该方法作为流水线的一部分，用于构建语法、语义及结构感知的基于中间表示的代码模型，以调优并行代码区域/内核。我们基于PolyBench、Rodinia、STREAM、DataRaceBench、AMD SDK、NPB、NVIDIA SDK、Parboil、SHOC及LULESH基准测试中的OpenMP与OpenCL代码区域/内核开展大量实验，将多模态学习技术应用于以下任务：（i）优化OpenMP循环中的线程数、调度策略及块大小；（ii）为OpenCL内核的异构设备映射确定最优设备。实验表明，该多模态学习方法在所有实验中均优于现有最优方案。