Scientific Machine Learning (SciML) is concerned with the development of learned emulators of physical systems governed by partial differential equations (PDE). In application domains such as weather forecasting, molecular dynamics, and inverse design, ML-based surrogate models are increasingly used to augment or replace inefficient and often non-differentiable numerical simulation algorithms. While a number of ML-based methods for approximating the solutions of PDEs have been proposed in recent years, they typically do not adapt to the parameters of the PDEs, making it difficult to generalize to PDE parameters not seen during training. We propose a Channel Attention mechanism guided by PDE Parameter Embeddings (CAPE) component for neural surrogate models and a simple yet effective curriculum learning strategy. The CAPE module can be combined with neural PDE solvers allowing them to adapt to unseen PDE parameters. The curriculum learning strategy provides a seamless transition between teacher-forcing and fully auto-regressive training. We compare CAPE in conjunction with the curriculum learning strategy using a popular PDE benchmark and obtain consistent and significant improvements over the baseline models. The experiments also show several advantages of CAPE, such as its increased ability to generalize to unseen PDE parameters without large increases inference time and parameter count.

翻译：科学机器学习（SciML）致力于开发能够模拟偏微分方程（PDE）所支配物理系统的学习型仿真器。在天气预报、分子动力学和逆向设计等应用领域，基于机器学习的替代模型正日益被用于增强或替代低效且通常不可微的数值模拟算法。尽管近年来已提出多种基于机器学习的方法来近似求解偏微分方程，但这些方法通常不能适应偏微分方程的参数变化，导致难以泛化到训练中未见的PDE参数。我们提出一种由PDE参数嵌入引导的通道注意力（CAPE）模块，用于神经替代模型，并引入一种简单而有效的课程学习策略。CAPE模块可与神经PDE求解器结合，使其能够适应未见过的PDE参数。该课程学习策略实现了教师强制训练与完全自回归训练之间的平滑过渡。我们在常用PDE基准测试上将CAPE与课程学习策略结合进行对比，较基线模型取得了一致且显著的性能提升。实验还展示了CAPE的多项优势，例如在不显著增加推理时间和参数量的情况下，其泛化到未见PDE参数的能力得到增强。