Global climate models (GCMs) are the main tools for understanding and predicting climate change. However, due to limited numerical resolutions, these models suffer from major structural uncertainties; e.g., they cannot resolve critical processes such as small-scale eddies in atmospheric and oceanic turbulence. Thus, such small-scale processes have to be represented as a function of the resolved scales via closures (parametrization). The accuracy of these closures is particularly important for capturing climate extremes. Traditionally, such closures are based on heuristics and simplifying assumptions about the unresolved physics. Recently, supervised-learned closures, trained offline on high-fidelity data, have been shown to outperform the classical physics-based closures. However, this approach requires a significant amount of high-fidelity training data and can also lead to instabilities. Reinforcement learning is emerging as a potent alternative for developing such closures as it requires only low-order statistics and leads to stable closures. In Scientific Multi-Agent Reinforcement Learning (SMARL) computational elements serve a dual role of discretization points and learning agents. We leverage SMARL and fundamentals of turbulence physics to learn closures for prototypes of atmospheric and oceanic turbulence. The policy is trained using only the enstrophy spectrum, which is nearly invariant and can be estimated from a few high-fidelity samples (these few samples are far from enough for supervised/offline learning). We show that these closures lead to stable low-resolution simulations that, at a fraction of the cost, can reproduce the high-fidelity simulations' statistics, including the tails of the probability density functions. The results demonstrate the high potential of SMARL for closure modeling for GCMs, especially in the regime of scarce data and indirect observations.

翻译：全球气候模型（GCM）是理解和预测气候变化的主要工具。然而，受限于数值分辨率不足，这些模型存在显著的结构性不确定性，例如无法解析大气与海洋湍流中的小尺度涡旋等关键过程。因此，这些微小尺度过程必须通过闭合方案（参数化）表示为可解析尺度的函数。这些闭合方案的准确性对于捕捉气候极端事件尤为重要。传统上，此类闭合方案基于对未解析物理过程的经验假设和简化处理。近年来，基于高保真数据离线训练的监督学习型闭合方案已被证明优于经典物理驱动方案。但该方法需要大量高保真训练数据，且可能引发数值不稳定性。强化学习正成为发展此类闭合方案的有力替代方案——它仅需低阶统计量即可得到稳定的闭合方案。在科学多智能体强化学习（SMARL）框架中，计算单元同时承担离散化网格点和学习代理的双重角色。我们结合SMARL与湍流物理基本原理，为大气与海洋湍流原型系统学习闭合方案。该策略仅使用近似守恒的涡度拟能谱进行训练，该谱可从少量高保真样本中估算（这些有限样本远不足以支撑监督/离线学习）。研究表明，这些闭合方案能产生稳定的低分辨率模拟，以较低计算代价复现高保真模拟的统计特征，包括概率密度函数的尾部形态。实验结果充分展示了SMARL在GCM闭合建模中的巨大潜力，尤其适用于数据稀缺和间接观测场景。