Machine learning for differential equations paves the way for computationally efficient alternatives to numerical solvers, with potentially broad impacts in science and engineering. Though current algorithms typically require simulated training data tailored to a given setting, one may instead wish to learn useful information from heterogeneous sources, or from real dynamical systems observations that are messy or incomplete. In this work, we learn general-purpose representations of PDEs from heterogeneous data by implementing joint embedding methods for self-supervised learning (SSL), a framework for unsupervised representation learning that has had notable success in computer vision. Our representation outperforms baseline approaches to invariant tasks, such as regressing the coefficients of a PDE, while also improving the time-stepping performance of neural solvers. We hope that our proposed methodology will prove useful in the eventual development of general-purpose foundation models for PDEs.

翻译：机器学习在微分方程中的应用为数值求解器提供了计算高效的替代方案，有望在科学与工程领域产生广泛影响。尽管现有算法通常需要针对特定场景定制的模拟训练数据，但研究人员更希望从异构数据源或真实动态系统观测（可能包含噪声或不完整信息）中学习有用的特征。本研究通过实现自监督学习（SSL）中的联合嵌入方法（该方法作为无监督表示学习框架，在计算机视觉领域已取得显著成功），从异构数据中学习偏微分方程的通用表示。我们的表示在回归PDE系数等不变性任务中优于基线方法，同时提升了神经求解器的时间步进性能。我们期望所提出的方法能为最终开发PDE通用基础模型提供助力。