Physics-Informed Neural Networks (PINNs) have emerged as a promising deep learning framework for approximating numerical solutions to partial differential equations (PDEs). However, conventional PINNs, relying on multilayer perceptrons (MLP), neglect the crucial temporal dependencies inherent in practical physics systems and thus fail to propagate the initial condition constraints globally and accurately capture the true solutions under various scenarios. In this paper, we introduce a novel Transformer-based framework, termed PINNsFormer, designed to address this limitation. PINNsFormer can accurately approximate PDE solutions by utilizing multi-head attention mechanisms to capture temporal dependencies. PINNsFormer transforms point-wise inputs into pseudo sequences and replaces point-wise PINNs loss with a sequential loss. Additionally, it incorporates a novel activation function, Wavelet, which anticipates Fourier decomposition through deep neural networks. Empirical results demonstrate that PINNsFormer achieves superior generalization ability and accuracy across various scenarios, including PINNs failure modes and high-dimensional PDEs. Moreover, PINNsFormer offers flexibility in integrating existing learning schemes for PINNs, further enhancing its performance.

翻译：物理信息神经网络（PINNs）已成为一种具有前景的深度学习框架，用于逼近偏微分方程（PDEs）的数值解。然而，依赖多层感知机（MLP）的传统PINNs忽视了实际物理系统中固有的关键时间依赖性，因此无法全局传播初始条件约束并在各种场景下准确捕获真实解。本文提出一种基于Transformer的新型框架——PINNsFormer，旨在解决这一局限。PINNsFormer通过利用多头注意力机制捕获时间依赖性，能够准确逼近PDE解。它将逐点输入转化为伪序列，并用序列损失替代逐点PINNs损失。此外，PINNsFormer引入新型激活函数Wavelet，通过深度神经网络预演傅里叶分解。实验结果表明，PINNsFormer在各种场景下（包括PINNs失效模式和高维PDEs）均展现出优越的泛化能力和准确性。同时，PINNsFormer能够灵活集成现有PINNs学习方案，进一步提升其性能。