The rapid development of large climate models has created the requirement of storing and transferring massive atmospheric data worldwide. Therefore, data compression is essential for meteorological research, but an efficient compression scheme capable of keeping high accuracy with high compressibility is still lacking. As an emerging technique, Implicit Neural Representation (INR) has recently acquired impressive momentum and demonstrates high promise for compressing diverse natural data. However, the INR-based compression encounters a bottleneck due to the sophisticated spatio-temporal properties and variability. To address this issue, we propose Hierarchical Harmonic decomposition implicit neural compression (HiHa) for atmospheric data. HiHa firstly segments the data into multi-frequency signals through decomposition of multiple complex harmonic, and then tackles each harmonic respectively with a frequency-based hierarchical compression module consisting of sparse storage, multi-scale INR and iterative decomposition sub-modules. We additionally design a temporal residual compression module to accelerate compression by utilizing temporal continuity. Experiments depict that HiHa outperforms both mainstream compressors and other INR-based methods in both compression fidelity and capabilities, and also demonstrate that using compressed data in existing data-driven models can achieve the same accuracy as raw data.
翻译:大型气候模型的快速发展对全球范围内大气数据的存储与传输提出了更高要求。因此,数据压缩对于气象研究至关重要,但目前仍缺乏一种能够在高压缩比下保持高精度的有效压缩方案。作为一种新兴技术,隐式神经表示(INR)近年来发展迅速,在多种自然数据压缩中展现出巨大潜力。然而,基于INR的压缩方法因大气数据复杂的时空特性与变异性而遇到瓶颈。为解决这一问题,我们提出了面向大气数据的分层谐波分解隐式神经压缩方法(HiHa)。HiHa首先通过多重复谐波分解将数据分割为多频信号,随后采用基于频率的分层压缩模块分别处理各谐波分量,该模块包含稀疏存储、多尺度INR与迭代分解子模块。我们还设计了一个时序残差压缩模块,利用时间连续性以加速压缩过程。实验表明,HiHa在压缩保真度与综合性能上均优于主流压缩器及其他基于INR的方法,并验证了使用压缩数据在现有数据驱动模型中能达到与原始数据相同的精度。