Weather forecasting is a long-standing computational challenge with direct societal and economic impacts. This task involves a large amount of continuous data collection and exhibits rich spatiotemporal dependencies over long periods, making it highly suitable for deep learning models. In this paper, we apply pre-training techniques to weather forecasting and propose W-MAE, a Weather model with Masked AutoEncoder pre-training for multi-variable weather forecasting. W-MAE is pre-trained in a self-supervised manner to reconstruct spatial correlations within meteorological variables. On the temporal scale, we fine-tune the pre-trained W-MAE to predict the future states of meteorological variables, thereby modeling the temporal dependencies present in weather data. We pre-train W-MAE using the fifth-generation ECMWF Reanalysis (ERA5) data, with samples selected every six hours and using only two years of data. Under the same training data conditions, we compare W-MAE with FourCastNet, and W-MAE outperforms FourCastNet in precipitation forecasting. In the setting where the training data is far less than that of FourCastNet, our model still performs much better in precipitation prediction (0.80 vs. 0.98). Additionally, experiments show that our model has a stable and significant advantage in short-to-medium-range forecasting (i.e., forecasting time ranges from 6 hours to one week), and the longer the prediction time, the more evident the performance advantage of W-MAE, further proving its robustness.

翻译：天气预报是一项长期存在的计算挑战，具有直接的社会和经济效益。该任务涉及大量连续数据收集，并表现出长期丰富的时空依赖性，因此非常适合深度学习模型。在本文中，我们将预训练技术应用于天气预报，并提出W-MAE，一种基于掩码自编码器预训练的多变量天气预报天气模型。W-MAE以自监督方式预训练，用于重建气象变量中的空间相关性。在时间尺度上，我们对预训练的W-MAE进行微调，以预测气象变量的未来状态，从而模拟天气数据中的时间依赖性。我们使用第五代ECMWF再分析（ERA5）数据预训练W-MAE，每六小时选取一次样本，且仅使用两年数据。在相同训练数据条件下，我们将W-MAE与FourCastNet进行比较，W-MAE在降水预报中优于FourCastNet。在训练数据远少于FourCastNet的设置下，我们的模型在降水预测中仍表现更优（0.80对比0.98）。此外，实验表明，我们的模型在中短期预报（即预报时间范围从6小时到一周）中具有稳定且显著的优势，且预测时间越长，W-MAE的性能优势越明显，进一步证明了其鲁棒性。