The ability to handle a large volume of data generated by scientific applications is crucial. We have seen an increase in the heterogeneity of storage technologies available to scientific applications, such as burst buffers, local temporary block storage, managed cloud parallel file systems (PFS), and non-POSIX object stores. However, scientific applications designed for traditional HPC systems can not easily exploit those storage systems due to cost, throughput, and programming model challenges. We present iFast, a new library-level approach to transparently accelerating scientific applications based on MPI-IO. It decouples application I/O, data caching, and data storage to support heterogeneous storage models. Design decisions of iFast are based on a strong emphasis on deployability. It is highly general with only MPI as a core dependency, allowing users to run unmodified MPI-based applications with unmodified MPI implementations - even proprietary ones like IntelMPI and Cray MPICH. Our approach supports a wide range of networked storage, including traditional PFS, ordinary NFS, and S3-based cloud storage. Unlike previous approaches, iFast ensures crash consistency even across compute nodes. We demonstrate iFast in cloud HPC platform, small local cluster, and hybrid of both to show its generality. Our results show that iFast reduces end-to-end execution time by 13-26% for three popular scientific applications on the cloud. It also outperforms the state-of-the-art system, SymphonyFS, a filesystem-based approach for similar goals but without crash consistency, by 12-23%.

翻译：处理科学应用生成的大量数据的能力至关重要。我们观察到科学应用可用的存储技术异构性日益增加，例如突发缓冲区、本地临时块存储、托管云并行文件系统（PFS）以及非POSIX对象存储。然而，为传统HPC系统设计的科学应用由于成本、吞吐量和编程模型等方面的挑战，难以充分利用这些存储系统。我们提出iFast，一种基于MPI-IO的透明加速科学应用的新型库级方法。它将应用I/O、数据缓存和数据存储解耦，以支持异构存储模型。iFast的设计决策高度强调可部署性。它具有高度的通用性，仅以MPI为核心依赖，允许用户运行未经修改的基于MPI的应用，同时使用未经修改的MPI实现——即使像IntelMPI和Cray MPICH这样的专有实现也不例外。我们的方法支持广泛的网络存储，包括传统PFS、普通NFS以及基于S3的云存储。与先前的方法不同，iFast即使在跨计算节点的情况下也能确保崩溃一致性。我们在云HPC平台、小型本地集群以及两者的混合环境中演示了iFast，以展示其通用性。结果表明，对于云上的三种流行科学应用，iFast将端到端执行时间减少了13-26%。它比现有最先进的系统SymphonyFS（一种基于文件系统的类似目标方法，但不具备崩溃一致性）性能提升了12-23%。