As the resolution of weather and climate simulations increases, the amount of data produced is growing rapidly from hundreds of terabytes to tens of petabytes. The huge size becomes a limiting factor for broader adoption, and its fast growth rate will soon exhaust all the available storage devices. To address these issues, we present EBCC (Error Bounded Climate-data Compressor). It follows a two-layer approach: a base compression layer using JPEG2000 to capture the bulk of the data with a high compression ratio, and a residual compression layer using wavelet transform and SPIHT encoding to efficiently eliminate long-tail extreme errors introduced by the base compression layer. It incorporates a feedback rate-control mechanism for both layers that adjusts compression ratios to achieve the specified maximum error target. We evaluate EBCC alongside other established compression methods on benchmarks related to weather and climate science including error statistics, a case study on primitive and derived variables near a hurricane, evaluation of the closure of the global energy budget, and a Lagrangian air parcel trajectory simulation. This is the first time that trajectory simulation is used to benchmark compression methods. Our method concentrates most errors near zero, while others tend to distribute errors uniformly within the error bound. EBCC outperforms other methods in the benchmarks at relative error targets ranging from 0.1% to 10% and achieves compression ratios from 15x to more than 300x. In the energy budget closure and Lagrangian trajectory benchmarks, it can achieve more than 100x compression while keeping errors within natural variability derived from ERA5 uncertainty members. This verifies the effectiveness of EBCC in creating heavily compressed weather and climate datasets suitable for downstream applications. The source code of EBCC is available in github.com/spcl/EBCC.

翻译：随着天气与气候模拟分辨率的提升，产生的数据量正从数百太字节迅速增长至数十拍字节。庞大的数据规模已成为其广泛应用的制约因素，而其快速增长速率将很快耗尽所有可用存储设备。为解决这些问题，我们提出EBCC（有界误差气候数据压缩器）。该方法采用双层架构：基础压缩层使用JPEG2000以高压缩比捕获数据主体，残差压缩层则通过小波变换与SPIHT编码有效消除基础压缩层引入的长尾极端误差。系统包含针对双层的反馈率控制机制，可动态调整压缩比以实现指定的最大误差目标。我们在天气与气候科学相关基准测试中评估EBCC及其他成熟压缩方法，评估内容包括误差统计、飓风附近原始变量与派生变量的案例研究、全球能量收支闭合性评估，以及拉格朗日空气质点轨迹模拟。本研究首次将轨迹模拟用于压缩方法基准测试。我们的方法使绝大多数误差集中在零值附近，而其他方法倾向于在误差界内均匀分布误差。在相对误差目标为0.1%至10%的基准测试中，EBCC以15倍至300倍以上的压缩比优于其他方法。在能量收支闭合与拉格朗日轨迹基准测试中，该方法可实现超过100倍的压缩比，同时将误差控制在ERA5不确定性成员推导的自然变率范围内。这验证了EBCC在生成适用于下游应用的高度压缩天气与气候数据集方面的有效性。EBCC源代码发布于github.com/spcl/EBCC。