Many large-scale scientific applications exhibit time-varying behavior, yet production HPC clusters still rely on rigid, fixed-size allocations, and most dynamic techniques remain confined to laboratory prototypes. This work presents a practical MPI malleability methodology that integrates with state-of-the-art high-performance computing (HPC) software stacks and operational practices. The methodology is implemented in the Dynamic Management of Resources (DMR) framework and is designed to ease adoption by existing applications without requiring intrusive code changes or scheduler modifications. We evaluate our approach by integrating the DMR API into two large-scale scientific applications and deploying them on three TOP500 supercomputers under realistic production configurations. Our non-invasive malleability solution achieves performance comparable to static baselines in controlled environments while substantially reducing node-hour consumption for identical workloads. These results show that malleability can be effectively exploited on production systems using vanilla resource managers, lowering the barrier to adoption of dynamic resource management in HPC.
翻译:许多大规模科学应用表现出随时间变化的行为,但生产级HPC集群仍依赖于僵化的固定大小资源分配,且大多数动态技术仍局限于实验室原型。本文提出了一种实用的MPI可伸缩性方法,该方法可与先进的高性能计算(HPC)软件栈及运维实践相集成。该方案通过动态资源管理(DMR)框架实现,旨在降低现有应用的采用门槛,无需侵入式代码修改或调度器调整。我们通过将DMR API集成至两个大规模科学应用,并在三台TOP500超级计算机上以真实生产配置进行部署,评估了该方法。在受控环境中,我们的非侵入式可伸缩性方案性能与静态基准相当,同时显著减少了相同工作负载的节点小时消耗。这些结果表明,使用标准资源管理器即可在生产系统上有效利用可伸缩性,从而降低HPC中动态资源管理的采用壁垒。