Effective intra-node GPU communication is essential for optimizing performance in MPI-based HPC applications, especially when leveraging multiple communication paths. In this study, we propose a novel approach that integrates CUDA Graphs into the UCX framework to enhance intra-node multi-path point-to-point GPU communication. By concurrently leveraging multiple paths, including NVLink and PCIe through the host, and optimizing communication workflows using CUDA Graph, we achieve significant reductions in communication overhead and improve execution efficiency. To the best of our knowledge, our proposed approach is the first to seamlessly integrate CUDA Graphs into UCX. Through extensive experiments on a four-GPU node, our proposed CUDA Graph-based multi-path communication approach achieves up to a 2.95x bandwidth improvement, compared to the single-path UCX (UCT::CUDA-IPC), in GPU-to-GPU OMB bandwidth test when utilizing the host path and two other GPU paths, at message sizes up to 512MB.

翻译：高效的节点内GPU通信对于优化基于MPI的高性能计算应用性能至关重要，特别是在利用多通信路径时。本研究提出了一种创新方法，将CUDA图集成到UCX框架中，以增强节点内多路径点对点GPU通信。通过同时利用包括NVLink和通过主机的PCIe在内的多条路径，并采用CUDA图优化通信工作流，我们显著降低了通信开销并提升了执行效率。据我们所知，所提出的方法是首个将CUDA图无缝集成到UCX中的方案。通过在四GPU节点上进行大量实验，与单路径UCX（UCT::CUDA-IPC）相比，基于CUDA图的多路径通信方法在GPU间OMB带宽测试中实现了高达2.95倍的带宽提升（在利用主机路径及另外两条GPU路径、消息大小最高达512MB时）。