With the approach of Exascale computing power for large-scale High Performance Computing (HPC) clusters, the gap between compute capabilities and storage systems is growing larger. This is particularly problematic for the Weather Research and Forecasting Model (WRF), a widely-used HPC application for high-resolution forecasting and research that produces sizable datasets, especially when analyzing transient weather phenomena. Despite this issue, the I/O modules within WRF have not been updated in the past ten years, resulting in subpar parallel I/O performance. This research paper demonstrates the positive impact of integrating ADIOS2, a next-generation parallel I/O framework, as a new I/O backend option in WRF. It goes into detail about the challenges encountered during the integration process and how they were addressed. The resulting I/O times show an over tenfold improvement when using ADIOS2 compared to traditional MPI-I/O based solutions. Furthermore, the study highlights the new features available to WRF users worldwide, such as the Sustainable Staging Transport (SST) enabling Unified Communication X (UCX) DataTransport, the node-local burst buffer write capabilities and in-line lossless compression capabilities of ADIOS2. Additionally, the research shows how ADIOS2's in-situ analysis capabilities can be smoothly integrated with a simple WRF forecasting pipeline, resulting in a significant improvement in overall time to solution. This study serves as a reminder to legacy HPC applications that incorporating modern libraries and tools can lead to considerable performance enhancements with minimal changes to the core application.

翻译：随着面向大规模高性能计算（HPC）集群的百亿亿次计算能力的到来，计算能力与存储系统之间的差距日益增大。这一问题对天气研究与预报模型（WRF）尤为突出——作为一款广泛应用于高分辨率天气预报和研究的HPC应用，WRF会生成海量数据集，尤其在分析瞬态天气现象时。然而，尽管存在这一挑战，WRF内部的I/O模块在过去十年中未曾更新，导致其并行I/O性能欠佳。本研究论文展示了集成下一代并行I/O框架ADIOS2作为WRF新I/O后端选项的积极影响。论文详细阐述了集成过程中遇到的挑战及其解决方案。实验结果表明，与传统的基于MPI-I/O的方案相比，使用ADIOS2后I/O时间提升了十倍以上。此外，本研究还强调了面向全球WRF用户的新功能，例如支持统一通信X（UCX）数据传输的可持续暂存传输（SST）、节点本地突发缓冲区写入能力以及ADIOS2的内联无损压缩功能。同时，研究展示了如何将ADIOS2的原位分析能力平滑集成到简单的WRF预报流程中，从而显著缩短整体求解时间。这项研究提醒传统HPC应用：通过集成现代库和工具，以最小化对核心应用的修改即可实现显著的性能提升。