Foundation models, such as large language models, have demonstrated success in addressing various language and image processing tasks. In this work, we introduce a multi-modal foundation model for scientific problems, named PROSE-PDE. Our model, designed for bi-modality to bi-modality learning, is a multi-operator learning approach which can predict future states of spatiotemporal systems while concurrently learning the underlying governing equations of the physical system. Specifically, we focus on multi-operator learning by training distinct one-dimensional time-dependent nonlinear constant coefficient partial differential equations, with potential applications to many physical applications including physics, geology, and biology. More importantly, we provide three extrapolation studies to demonstrate that PROSE-PDE can generalize physical features through the robust training of multiple operators and that the proposed model can extrapolate to predict PDE solutions whose models or data were unseen during the training. Furthermore, we show through systematic numerical experiments that the utilization of the symbolic modality in our model effectively resolves the well-posedness problems with training multiple operators and thus enhances our model's predictive capabilities.

翻译：基础模型，如大型语言模型，已在处理各种语言和图像任务中展现出成功。在本工作中，我们针对科学问题引入了一种多模态基础模型，命名为PROSE-PDE。我们的模型专为双模态到双模态学习设计，采用多算子学习方法，能够预测时空系统的未来状态，同时并行学习物理系统的基本控制方程。具体而言，我们专注于通过训练不同的单变量时间相关非线性常系数偏微分方程来实现多算子学习，这些方程在物理、地质和生物学等领域具有潜在应用。更重要的是，我们提供了三项外推研究，以证明PROSE-PDE可通过多算子的稳健训练泛化物理特征，并且该模型能够外推预测在训练中未见其模型或数据的PDE解。此外，通过系统数值实验，我们展示了模型中符号模态的利用有效解决了多算子训练时的适定性问题，从而增强了模型的预测能力。