Modern scientific applications predominantly run on large-scale computing platforms, necessitating collaboration between scientific domain experts and high-performance computing (HPC) experts. While domain experts are often skilled in customizing domain-specific scientific computing routines, which often involves various matrix computations, HPC experts are essential for achieving efficient execution of these computations on large-scale platforms. This process often involves utilizing complex parallel computing libraries tailored to specific matrix computation scenarios. However, the intricate programming procedure and the need for deep understanding in both application domains and HPC poses significant challenges to the widespread adoption of scientific computing. In this research, we observe that matrix computations can be transformed into equivalent graph representations, and that by utilizing graph processing engines, HPC experts can be freed from the burden of implementing efficient scientific computations. Based on this observation, we introduce a graph engine-based scientific computing (Graph for Science) paradigm, which provides a unified graph programming interface, enabling domain experts to promptly implement various types of matrix computations. The proposed paradigm leverages the underlying graph processing engine to achieve efficient execution, eliminating the needs for HPC expertise in programming large-scale scientific applications. Our results show that the graph engine-based scientific computing paradigm achieves performance comparable to the best-performing implementations based on existing parallel computing libraries and bespoke implementations. Importantly, the paradigm greatly simplifies the development of scientific computations on large-scale platforms, reducing the programming difficulty for scientists and facilitating broader adoption of scientific computing.
翻译:现代科学应用主要运行在大规模计算平台上,需要科学领域专家与高性能计算(HPC)专家协作。领域专家通常擅长定制特定领域的科学计算例程(常涉及多种矩阵计算),而高性能计算专家则负责确保这些计算在大规模平台上高效执行,这通常需要利用针对特定矩阵计算场景的复杂并行计算库。然而,复杂的编程流程以及对应用领域和高性能计算的深入理解需求,给科学计算的广泛普及带来了重大挑战。在本研究中,我们观察到矩阵计算可以转化为等价的图表示,通过利用图处理引擎,高性能计算专家可以摆脱实现高效科学计算的重担。基于这一观察,我们提出了一种基于图引擎的科学计算范式(科学计算图),该范式提供统一的图编程接口,使领域专家能够快速实现各种矩阵计算。所提出的范式利用底层图处理引擎实现高效执行,消除了在编写大规模科学应用时对高性能计算专业知识的需求。我们的结果表明,基于图引擎的科学计算范式达到了与基于现有并行计算库的最优实现以及定制实现相当的性能。重要的是,该范式极大简化了在大规模平台上开发科学计算的过程,降低了科学家的编程难度,促进了科学计算的更广泛普及。