Recent years have witnessed the promise of coupling machine learning methods and physical domain-specific insight for solving scientific problems based on partial differential equations (PDEs). However, being data-intensive, these methods still require a large amount of PDE data. This reintroduces the need for expensive numerical PDE solutions, partially undermining the original goal of avoiding these expensive simulations. In this work, seeking data efficiency, we design unsupervised pretraining and in-context learning methods for PDE operator learning. To reduce the need for training data with simulated solutions, we pretrain neural operators on unlabeled PDE data using reconstruction-based proxy tasks. To improve out-of-distribution performance, we further assist neural operators in flexibly leveraging in-context learning methods, without incurring extra training costs or designs. Extensive empirical evaluations on a diverse set of PDEs demonstrate that our method is highly data-efficient, more generalizable, and even outperforms conventional vision-pretrained models.

翻译：近年来，将机器学习方法与物理领域专业知识相结合，为求解基于偏微分方程的科学问题展现了巨大潜力。然而，这些方法通常需要大量偏微分方程数据，重新引入了对昂贵数值求解的依赖，部分背离了最初避免这些高成本仿真的目标。针对数据效率问题，本工作设计了面向偏微分方程算子学习的无监督预训练与上下文学习方法。为减少对带仿真解的训练数据的需求，我们通过基于重构的代理任务在无标签的偏微分方程数据上预训练神经算子；为提升分布外性能，我们进一步辅助神经算子灵活利用上下文学习方法，且无需额外的训练成本或模型设计。在多种偏微分方程上的大量实证评估表明：我们的方法具有高度数据效率、更强的泛化能力，甚至超越了传统视觉预训练模型。