With the ever-increasing computing power of supercomputers and the growing scale of scientific applications, the efficiency of MPI collective communications turns out to be a critical bottleneck in large-scale distributed and parallel processing. Large message size in MPI collectives is a particularly big concern because it may significantly delay the overall parallel performance. To address this issue, prior research simply applies the off-the-shelf fix-rate lossy compressors in the MPI collectives, leading to suboptimal performance, limited generalizability, and unbounded errors. In this paper, we propose a novel solution, called C-Coll, which leverages error-bounded lossy compression to significantly reduce the message size, resulting in a substantial reduction in communication cost. The key contributions are three-fold. (1) We develop two general, optimized lossy-compression-based frameworks for both types of MPI collectives (collective data movement as well as collective computation), based on their particular characteristics. Our framework not only reduces communication cost but also preserves data accuracy. (2) We customize an optimized version based on SZx, an ultra-fast error-bounded lossy compressor, which can meet the specific needs of collective communication. (3) We integrate C-Coll into multiple collectives, such as MPI_Allreduce, MPI_Scatter, and MPI_Bcast, and perform a comprehensive evaluation based on real-world scientific datasets. Experiments show that our solution outperforms the original MPI collectives as well as multiple baselines and related efforts by 3.5-9.7X.

翻译：摘要：随着超级计算机计算能力的持续提升与科学应用规模的不断扩大，MPI集合通信的效率已成为大规模分布式与并行处理中的关键瓶颈。MPI集合操作中的大消息尺寸尤其值得关注，因其可能显著拖累整体并行性能。为应对此问题，先前研究直接在MPI集合操作中应用现成定速率有损压缩器，导致性能欠优、泛化能力有限且误差无界。本文提出一种创新解决方案C-Coll，通过利用误差有界有损压缩显著缩减消息尺寸，从而大幅降低通信开销。核心贡献包含三个方面：（1）针对两类MPI集合操作（集合数据移动与集合计算）的各自特性，分别开发两种通用且优化的基于有损压缩的框架。该框架不仅降低通信成本，同时保障数据精度。（2）基于超快速误差有界有损压缩器SZx定制优化版本，以契合集合通信的特定需求。（3）将C-Coll集成至MPI_Allreduce、MPI_Scatter、MPI_Bcast等多种集合操作中，并基于真实科学数据集开展全面评估。实验表明，我们的方案相较原始MPI集合操作及多种基线方法与相关工作，性能提升达3.5-9.7倍。