Today's scientific simulations, for example in the high-performance exascale sector, produce huge amounts of data. Due to limited I/O bandwidth and available storage space, there is the necessity to reduce scientific data of high performance computing applications. Error-bounded lossy compression has been proven to be an effective approach tackling the trade-off between accuracy and storage space. Within this work, we are exploring and discussing error-bounded lossy compression solely based on adaptive mesh refinement techniques. This compression technique is not only easily integrated into existing adaptive mesh refinement applications but also suits as a general lossy compression approach for arbitrary data in form of multi-dimensional arrays, irrespective of the data type. Moreover, these techniques permit the exclusion of regions of interest and even allows for nested error domains during the compression. The described data compression technique is presented exemplary on ERA5 data.
翻译:当前的科学模拟,例如在高性能百亿亿次计算领域,会产生海量数据。由于有限的I/O带宽和可用存储空间,有必要对高性能计算应用产生的科学数据进行缩减。误差有界有损压缩已被证明是解决精度与存储空间之间权衡的有效方法。本工作中,我们探索并讨论一种完全基于自适应网格细化技术的误差有界有损压缩方法。该压缩技术不仅易于集成到现有的自适应网格细化应用中,也适用于任意多维数组形式数据的通用有损压缩,且不受数据类型限制。此外,这些技术允许在压缩过程中排除感兴趣区域,甚至支持嵌套误差域。所述数据压缩技术以ERA5数据为例进行了展示。