Neural solvers for partial differential equations (PDEs) have great potential, yet their practicality is currently limited by their generalizability. PDEs evolve over broad scales and exhibit diverse behaviors; predicting these phenomena will require learning representations across a wide variety of inputs, which may encompass different coefficients, geometries, or equations. As a step towards generalizable PDE modeling, we adapt masked pretraining for PDEs. Through self-supervised learning across PDEs, masked autoencoders can learn useful latent representations for downstream tasks. In particular, masked pretraining can improve coefficient regression and timestepping performance of neural solvers on unseen equations. We hope that masked pretraining can emerge as a unifying method across large, unlabeled, and heterogeneous datasets to learn latent physics at scale.

翻译：偏微分方程（PDE）的神经求解器具有巨大潜力，但其普适性目前仍受限于泛化能力。PDEs在广泛尺度上演化并呈现多样化行为；预测这些现象需要学习涵盖不同系数、几何形状或方程形式的多样化输入表征。为实现可泛化的PDE建模，我们针对PDEs调整了掩码预训练方法。通过跨PDEs的自监督学习，掩码自编码器能够为下游任务学习有效的潜在表征。特别地，掩码预训练可提升神经求解器在未见方程上的系数回归与时间步进性能。我们期望掩码预训练能成为统一方法，在大规模、无标签、异构数据集中实现潜在物理规律的学习。