Aggregated HPC resources have rigid allocation systems and programming models which struggle to adapt to diverse and changing workloads. Consequently, HPC systems fail to efficiently use the large pools of unused memory and increase the utilization of idle computing resources. Prior work attempted to increase the throughput and efficiency of supercomputing systems through workload co-location and resource disaggregation. However, these methods fall short of providing a solution that can be applied to existing systems without major hardware modifications and performance losses. In this paper, we improve the utilization of supercomputers by employing the new cloud paradigm of serverless computing. We show how serverless functions provide fine-grained access to the resources of batch-managed cluster nodes. We present an HPC-oriented Function-as-a-Service (FaaS) that satisfies the requirements of high-performance applications. We demonstrate a \emph{software resource disaggregation} approach where placing functions on unallocated and underutilized nodes allows idle cores and accelerators to be utilized while retaining near-native performance.
翻译:聚合式高性能计算资源具有僵化的分配系统和编程模型,难以适应多样化且动态变化的工作负载。因此,高性能计算系统无法有效利用大量闲置内存池,也无法提高空闲计算资源的利用率。先前的工作尝试通过工作负载共置和资源解耦来提升超算系统的吞吐量与效率,但这些方法未能提供一种无需重大硬件改造且无性能损失即可应用于现有系统的解决方案。本文通过采用无服务器计算这一新型云范式来提高超级计算机的利用率。我们展示了无服务器函数如何提供对批处理集群节点资源的细粒度访问,并提出了一种面向高性能计算的功能即服务(FaaS)架构,以满足高性能应用的需求。我们展示了一种**软件资源解耦**方法:将函数部署到未分配和利用不足的节点上,使空闲核心与加速器得到利用,同时保持接近本机的性能。