Neural operators have recently grown in popularity as Partial Differential Equation (PDEs) surrogate models. Learning solution functionals, rather than functions, has proven to be a powerful approach to calculate fast, accurate solutions to complex PDEs. While much work has been done evaluating neural operator performance on a wide variety of surrogate modeling tasks, these works normally evaluate performance on a single equation at a time. In this work, we develop a novel contrastive pretraining framework utilizing Generalized Contrastive Loss that improves neural operator generalization across multiple governing equations simultaneously. Governing equation coefficients are used to measure ground-truth similarity between systems. A combination of physics-informed system evolution and latent-space model output are anchored to input data and used in our distance function. We find that physics-informed contrastive pretraining improves both accuracy and generalization for the Fourier Neural Operator in fixed-future task, with comparable performance on the autoregressive rollout, and superresolution tasks for the 1D Heat, Burgers', and linear advection equations.

翻译：神经算子作为偏微分方程代理模型近年来日益流行。学习解泛函（而非函数）已被证明是快速准确求解复杂偏微分方程的有效方法。尽管已有大量研究评估神经算子在多种代理建模任务中的表现，但这些工作通常仅针对单个方程进行性能评估。本研究提出了一种利用广义对比损失的新型对比预训练框架，该框架可同时提升神经算子在多个控制方程上的泛化能力。通过使用控制方程系数衡量系统间的真实相似度，我们将物理信息驱动的系统演化与潜空间模型输出相结合，作为输入数据的锚点并融入距离函数中。实验表明，对于傅里叶神经算子，物理信息对比预训练在固定未来任务中提升了准确性与泛化能力，在自回归滚动预测和超分辨率任务中表现相当，适用于一维热传导方程、Burgers方程及线性平流方程。