Neural operators have proven to be a promising approach for modeling spatiotemporal systems in the physical sciences. However, training these models for large systems can be quite challenging as they incur significant computational and memory expense -- these systems are often forced to rely on autoregressive time-stepping of the neural network to predict future temporal states. While this is effective in managing costs, it can lead to uncontrolled error growth over time and eventual instability. We analyze the sources of this autoregressive error growth using prototypical neural operator models for physical systems and explore ways to mitigate it. We introduce architectural and application-specific improvements that allow for careful control of instability-inducing operations within these models without inflating the compute/memory expense. We present results on several scientific systems that include Navier-Stokes fluid flow, rotating shallow water, and a high-resolution global weather forecasting system. We demonstrate that applying our design principles to prototypical neural networks leads to significantly lower errors in long-range forecasts with 800\% longer forecasts without qualitative signs of divergence compared to the original models for these systems. We open-source our \href{https://anonymous.4open.science/r/stabilizing_neural_operators-5774/}{code} for reproducibility.

翻译：神经算子已被证明是模拟物理科学中时空系统的一种有前途的方法。然而，针对大型系统训练这些模型颇具挑战性，因为它们会带来巨大的计算和内存开销——这些系统通常不得不依赖神经网络的自回归时间步进来预测未来的时间状态。虽然这在控制成本方面行之有效，但可能导致随时间推移的不可控误差增长并最终引发不稳定性。我们利用用于物理系统的典型神经算子模型分析了这种自回归误差增长的根源，并探索了缓解方法。我们引入了架构层面和特定应用层面的改进，使得能够在计算/内存开销不增加的情况下精确控制这些模型内诱发不稳定的操作。我们展示了在多个科学系统上的结果，包括纳维-斯托克斯流体流动、旋转浅水模型以及一个高分辨率全球天气预报系统。我们证明，将我们的设计原则应用于原型神经网络，可使长期预测的误差显著降低，与原始模型相比，预测时长延长了800%且未出现发散迹象。我们开源了代码以供复现。