Solving Partial Differential Equations (PDEs) is the core of many fields of science and engineering. While classical approaches are often prohibitively slow, machine learning models often fail to incorporate complete system information. Over the past few years, transformers have had a significant impact on the field of Artificial Intelligence and have seen increased usage in PDE applications. However, despite their success, transformers currently lack integration with physics and reasoning. This study aims to address this issue by introducing PITT: Physics Informed Token Transformer. The purpose of PITT is to incorporate the knowledge of physics by embedding partial differential equations (PDEs) into the learning process. PITT uses an equation tokenization method to learn an analytically-driven numerical update operator. By tokenizing PDEs and embedding partial derivatives, the transformer models become aware of the underlying knowledge behind physical processes. To demonstrate this, PITT is tested on challenging PDE neural operators in both 1D and 2D prediction tasks. The results show that PITT outperforms the popular Fourier Neural Operator and has the ability to extract physically relevant information from governing equations.

翻译：求解偏微分方程是许多科学与工程领域的核心。尽管经典方法往往计算速度过慢，而机器学习模型又经常无法整合完整的系统信息。过去数年间，变换器在人工智能领域产生了重大影响，并在偏微分方程应用中得到了越来越多的使用。然而，尽管取得了成功，变换器目前仍缺乏与物理知识和推理的融合。本研究旨在通过引入"物理信息令牌变换器"来解决这一问题。PITT的目标是通过将偏微分方程嵌入学习过程中来融入物理知识。PITT使用一种方程令牌化方法，学习一个基于解析驱动的数值更新算子。通过对偏微分方程进行令牌化并嵌入偏导数，变换器模型能够感知物理过程背后的潜在知识。为验证这一点，我们在一维和二维预测任务中对PITT在具有挑战性的偏微分方程神经算子上进行了测试。结果表明，PITT性能优于流行的傅里叶神经算子，并且具备从控制方程中提取物理相关信息的能力。