Neural operators have proven to be a promising approach for modeling spatiotemporal systems in the physical sciences. However, training these models for large systems can be quite challenging as they incur significant computational and memory expense -- these systems are often forced to rely on autoregressive time-stepping of the neural network to predict future temporal states. While this is effective in managing costs, it can lead to uncontrolled error growth over time and eventual instability. We analyze the sources of this autoregressive error growth using prototypical neural operator models for physical systems and explore ways to mitigate it. We introduce architectural and application-specific improvements that allow for careful control of instability-inducing operations within these models without inflating the compute/memory expense. We present results on several scientific systems that include Navier-Stokes fluid flow, rotating shallow water, and a high-resolution global weather forecasting system. We demonstrate that applying our design principles to neural operators leads to significantly lower errors for long-term forecasts as well as longer time horizons without qualitative signs of divergence compared to the original models for these systems. We open-source our \href{https://github.com/mikemccabe210/stabilizing_neural_operators}{code} for reproducibility.

翻译：神经算子已被证明是物理科学中建模时空系统的一种有前途的方法。然而，针对大规模系统训练这些模型可能极具挑战性，因为它们会带来显著的计算和内存开销——这些系统往往被迫依赖神经网络的自回归时间步进来预测未来时间状态。尽管这在控制成本方面行之有效，但随着时间的推移，它可能导致不受控制的误差增长并最终引发不稳定性。我们使用物理系统的原型神经算子模型分析了这种自回归误差增长的根源，并探索了缓解方法。我们引入了架构和应用层面的改进措施，从而能够在不增加计算/内存开销的情况下，精细控制这些模型中引发不稳定性的操作。我们在多个科学系统上展示了结果，包括纳维-斯托克斯流体流动、旋转浅水模型以及一个高分辨率全球天气预报系统。我们证明，与原始模型相比，将我们的设计原则应用于神经算子可在长期预测中实现显著更低的误差，并延长无定性发散迹象的时间范围。我们开源了相关代码（https://github.com/mikemccabe210/stabilizing_neural_operators）以确保可复现性。