Solving Partial Differential Equations (PDEs) is the core of many fields of science and engineering. While classical approaches are often prohibitively slow, machine learning models often fail to incorporate complete system information. Over the past few years, transformers have had a significant impact on the field of Artificial Intelligence and have seen increased usage in PDE applications. However, despite their success, transformers currently lack integration with physics and reasoning. This study aims to address this issue by introducing PITT: Physics Informed Token Transformer. The purpose of PITT is to incorporate the knowledge of physics by embedding partial differential equations (PDEs) into the learning process. PITT uses an equation tokenization method to learn an analytically-driven numerical update operator. By tokenizing PDEs and embedding partial derivatives, the transformer models become aware of the underlying knowledge behind physical processes. To demonstrate this, PITT is tested on challenging 1D and 2D PDE neural operator prediction tasks. The results show that PITT outperforms popular neural operator models and has the ability to extract physically relevant information from governing equations.

翻译：求解偏微分方程是众多科学与工程领域的核心任务。经典方法往往计算成本极高，而机器学习模型又常无法整合完整的系统信息。近年来，变换器模型在人工智能领域产生了深远影响，并逐渐被应用于偏微分方程求解。然而，尽管取得了成功，当前变换器仍缺乏与物理规律和推理的深度融合。本研究旨在通过提出物理信息令牌变换器解决这一问题。PITT的核心思想是将偏微分方程嵌入学习过程中，从而融入物理知识。该模型采用方程令牌化方法，学习一个受解析驱动的数值更新算子。通过将偏微分方程进行令牌化并嵌入偏导数信息，变换器模型能够感知物理过程背后的潜在知识。为验证有效性，我们在具有挑战性的一维和二维偏微分方程神经算子预测任务上测试了PITT。结果表明，PITT不仅优于主流神经算子模型，而且具备从控制方程中提取物理相关信息的强大能力。