Remarkable progress in the development of Deep Learning Weather Prediction (DLWP) models positions them to become competitive with traditional numerical weather prediction (NWP) models. Indeed, a wide number of DLWP architectures -- based on various backbones, including U-Net, Transformer, Graph Neural Network (GNN), and Fourier Neural Operator (FNO) -- have demonstrated their potential at forecasting atmospheric states. However, due to differences in training protocols, forecast horizons, and data choices, it remains unclear which (if any) of these methods and architectures are most suitable for weather forecasting and for future model development. Here, we step back and provide a detailed empirical analysis, under controlled conditions, comparing and contrasting the most prominent DLWP models, along with their backbones. We accomplish this by predicting synthetic two-dimensional incompressible Navier-Stokes and real-world global weather dynamics. In terms of accuracy, memory consumption, and runtime, our results illustrate various tradeoffs. For example, on synthetic data, we observe favorable performance of FNO; and on the real-world WeatherBench dataset, our results demonstrate the suitability of ConvLSTM and SwinTransformer for short-to-mid-ranged forecasts. For long-ranged weather rollouts of up to 365 days, we observe superior stability and physical soundness in architectures that formulate a spherical data representation, i.e., GraphCast and Spherical FNO. In addition, we observe that all of these model backbones ``saturate,'' i.e., none of them exhibit so-called neural scaling, which highlights an important direction for future work on these and related models.

翻译：深度学习天气预测（DLWP）模型的发展取得了显著进展，使其有望与传统数值天气预报（NWP）模型相竞争。事实上，基于多种骨干网络（包括U-Net、Transformer、图神经网络（GNN）和傅里叶神经算子（FNO））的大量DLWP架构已展现出预测大气状态的潜力。然而，由于训练方案、预测时程和数据选择的差异，目前尚不清楚这些方法与架构中哪些（若有）最适合天气预报及未来模型开发。本文通过受控条件下的详细实证分析，比较并对比了最主流的DLWP模型及其骨干网络。我们通过预测合成的二维不可压缩纳维-斯托克斯方程和真实全球天气动力学数据实现这一目标。在精度、内存消耗和运行时间方面，我们的结果揭示了多种权衡关系。例如，在合成数据上，我们观察到FNO表现出优越性能；在真实世界的WeatherBench数据集上，结果显示ConvLSTM和SwinTransformer适合中短期预报。对于长达365天的长期天气推演，我们发现在采用球面数据表示的架构（即GraphCast和球面FNO）中具有更优的稳定性和物理合理性。此外，我们观察到所有这些模型骨干网络均存在“饱和”现象——即均未表现出所谓的神经缩放规律，这为这些模型及相关研究的未来工作指明了重要方向。