As supercomputers become larger with powerful Graphics Processing Unit (GPU), traditional direct eigensolvers struggle to keep up with the hardware evolution and scale efficiently due to communication and synchronization demands. Conversely, subspace eigensolvers, like the Chebyshev Accelerated Subspace Eigensolver (ChASE), have a simpler structure and can overcome communication and synchronization bottlenecks. ChASE is a modern subspace eigensolver that uses Chebyshev polynomials to accelerate the computation of extremal eigenpairs of dense Hermitian eigenproblems. In this work we show how we have modified ChASE by rethinking its memory layout, introducing a novel parallelization scheme, switching to a more performing communication-avoiding algorithm for one of its inner modules, and substituting the MPI library by the vendor-optimized NCCL library. The resulting library can tackle dense problems with size up to $N=\mathcal{O}(10^6)$, and scales effortlessly up to the full 900 nodes -- each one powered by 4$\times$A100 NVIDIA GPUs -- of the JUWELS Booster hosted at the J\"ulich Supercomputing Centre.
翻译:随着超级计算机规模不断扩大并配备强大的图形处理单元(GPU),传统的直接特征值求解器因其通信与同步需求,难以跟上硬件演进的步伐并实现高效扩展。相比之下,子空间特征值求解器(如切比雪夫加速子空间特征值求解器,ChASE)结构更为简单,能够突破通信与同步瓶颈。ChASE是一种现代子空间特征值求解器,利用切比雪夫多项式加速计算稠密埃尔米特特征问题的极端特征对。本文通过重新设计ChASE的内存布局、引入新型并行化方案、为其内部模块切换至性能更优的通信避免算法,以及将MPI库替换为供应商优化的NCCL库,展示了如何对ChASE进行改进。由此生成的库能够求解规模达$N=\mathcal{O}(10^6)$的稠密问题,并在位于于利希超级计算中心的JUWELS Booster集群上,轻松扩展至全部900个节点——每个节点配备4块NVIDIA A100 GPU。