Weather and climate simulations produce petabytes of high-resolution data that are later analyzed by researchers in order to understand climate change or severe weather. We propose a new method of compressing this multidimensional weather and climate data: a coordinate-based neural network is trained to overfit the data, and the resulting parameters are taken as a compact representation of the original grid-based data. While compression ratios range from 300x to more than 3,000x, our method outperforms the state-of-the-art compressor SZ3 in terms of weighted RMSE, MAE. It can faithfully preserve important large scale atmosphere structures and does not introduce artifacts. When using the resulting neural network as a 790x compressed dataloader to train the WeatherBench forecasting model, its RMSE increases by less than 2%. The three orders of magnitude compression democratizes access to high-resolution climate data and enables numerous new research directions.
翻译:气象与气候模拟生成PB级高分辨率数据,研究人员后续需分析这些数据以理解气候变化或极端天气事件。我们提出一种新型多维气象与气候数据压缩方法:通过训练基于坐标的神经网络过拟合数据,将所得参数作为原始网格数据的紧凑表示。在300倍至3000倍以上的压缩比范围内,该方法在加权RMSE和MAE指标上均优于当前最先进压缩器SZ3。该方法能忠实保留重要的大尺度大气结构,且不引入伪影。当使用该神经网络作为790倍压缩数据加载器训练WeatherBench预报模型时,其RMSE增长不足2%。这种三个数量级的压缩能力实现了高分辨率气候数据的大众化获取,并催生了大量新的研究方向。