Machine learning for differential equations paves the way for computationally efficient alternatives to numerical solvers, with potentially broad impacts in science and engineering. Though current algorithms typically require simulated training data tailored to a given setting, one may instead wish to learn useful information from heterogeneous sources, or from real dynamical systems observations that are messy or incomplete. In this work, we learn general-purpose representations of PDEs from heterogeneous data by implementing joint embedding methods for self-supervised learning (SSL), a framework for unsupervised representation learning that has had notable success in computer vision. Our representation outperforms baseline approaches to invariant tasks, such as regressing the coefficients of a PDE, while also improving the time-stepping performance of neural solvers. We hope that our proposed methodology will prove useful in the eventual development of general-purpose foundation models for PDEs. Code: https://github.com/facebookresearch/SSLForPDEs.

翻译：面向微分方程的机器学习为数值求解器提供了计算高效的替代方案，有望在科学与工程领域产生广泛影响。尽管现有算法通常需要针对特定场景定制的模拟训练数据，但我们或许更希望从异质数据源或不完整、杂乱的真实动力学系统观测中学习有用信息。本研究通过实现自监督学习（SSL）的联合嵌入方法，从异质数据中学习偏微分方程（PDE）的通用表征——这一无监督表征学习框架已在计算机视觉领域取得显著成功。我们的表征在不变性任务（如回归PDE系数）上优于基线方法，同时提升了神经求解器的时间步进性能。我们期望所提出的方法能为最终开发偏微分方程通用基础模型提供助力。代码：https://github.com/facebookresearch/SSLForPDEs