As we have entered Exascale computing, the faults in high-performance systems are expected to increase considerably. To compensate for a higher failure rate, the standard checkpoint/restart technique would need to create checkpoints at a much higher frequency resulting in an excessive amount of overhead which would not be sustainable for many scientific applications. Replication allows for fast recovery from failures by simply dropping the failed processes and using their replicas to continue the regular operation of the application. In this paper, we have implemented PartRePer-MPI, a novel fault-tolerant MPI library that adopts partial replication of some of the launched MPI processes in order to provide resilience from failures. The novelty of our work is that it combines both fault tolerance, due to the use of the User Level Failure Mitigation (ULFM) framework in the Open MPI library, and high performance, due to the use of communication protocols in the native MPI library that is generally fine-tuned for specific HPC platforms. We have implemented efficient and parallel communication strategies with computational and replica processes, and our library can seamlessly provide fault tolerance support to an existing MPI application. Our experiments using seven NAS Parallel Benchmarks and two scientific applications show that the failure-free overheads in PartRePer-MPI when compared to the baseline MVAPICH2, are only up to 6.4% for the NAS parallel benchmarks and up to 9.7% for the scientific applications.

翻译：随着我们进入百亿亿次计算时代，高性能系统中的故障预计将大幅增加。为应对更高的故障率，标准的检查点/重启技术需要以更高频率创建检查点，这将导致过高的开销，使得许多科学应用难以持续运行。复制技术通过直接丢弃故障进程并利用其副本继续执行应用程序的正常操作，实现了故障的快速恢复。本文实现了PartRePer-MPI，一种新型的容错MPI库，该库采用对部分已启动MPI进程的复制策略以提供故障恢复能力。本研究的创新之处在于：一方面通过Open MPI库中的用户级故障缓解（ULFM）框架实现容错，另一方面利用原生MPI库中针对特定HPC平台精细调优的通信协议实现高性能。我们针对计算进程与副本进程设计了高效并行通信策略，该库可无缝为现有MPI应用提供容错支持。基于七个NAS并行基准测试程序和两个科学应用的实验表明，与基线MVAPICH2相比，PartRePer-MPI在无故障情况下的开销分别为：NAS并行基准测试不超过6.4%，科学应用不超过9.7%。