In this short paper, we present an innovative approach to limit the required bandwidth when transferring data during in transit analysis. This approach is called hybrid because it combines existing in situ and in transit solutions. It leverages the stable ABI of Catalyst version 2 and the Catalyst-ADIOS2 implementation to seamlessly switch from in situ, in transit and hybrid analysis without modifying the numerical simulation code. The typical use case is to perform data reduction in situ then generate a visualization in transit on the reduced data. This approach makes the numerical simulation workflows very flexible depending on the size of the data, the available computing resources or the analysis type. Our experiment with this hybrid approach, reducing data before sending it, demonstrated large cost reductions for some visualization pipelines compared to in situ and in transit solutions. The implementation is available under an open source permissive license to be usable broadly in any scientific community.
翻译:在这篇短文中,我们提出了一种创新方法,用于限制传输分析过程中数据传输所需的带宽。该方法被称为混合方法,因为它结合了现有的原位和传输解决方案。它利用Catalyst 2.0版本的稳定ABI以及Catalyst-ADIOS2实现,无需修改数值模拟代码即可无缝切换原位分析、传输分析及混合分析。典型用例是先在原位执行数据缩减,随后在缩减后的数据上进行传输可视化。这种方法使得数值模拟工作流能够根据数据规模、可用计算资源或分析类型实现高度灵活性。我们采用这种混合方法进行的实验表明,在发送数据前先进行缩减,相较于纯原位和纯传输方案,某些可视化流程可大幅降低成本。该实现采用开源许可协议发布,可在任何科学领域广泛使用。