Physics-Informed Neural Networks (PINNs) have emerged as a promising deep learning framework for approximating numerical solutions to partial differential equations (PDEs). However, conventional PINNs, relying on multilayer perceptrons (MLP), neglect the crucial temporal dependencies inherent in practical physics systems and thus fail to propagate the initial condition constraints globally and accurately capture the true solutions under various scenarios. In this paper, we introduce a novel Transformer-based framework, termed PINNsFormer, designed to address this limitation. PINNsFormer can accurately approximate PDE solutions by utilizing multi-head attention mechanisms to capture temporal dependencies. PINNsFormer transforms point-wise inputs into pseudo sequences and replaces point-wise PINNs loss with a sequential loss. Additionally, it incorporates a novel activation function, Wavelet, which anticipates Fourier decomposition through deep neural networks. Empirical results demonstrate that PINNsFormer achieves superior generalization ability and accuracy across various scenarios, including PINNs failure modes and high-dimensional PDEs. Moreover, PINNsFormer offers flexibility in integrating existing learning schemes for PINNs, further enhancing its performance.

翻译：物理信息神经网络（PINNs）已成为一种有前景的深度学习框架，用于近似求解偏微分方程（PDEs）的数值解。然而，传统的基于多层感知机（MLP）的PINNs忽略了实际物理系统中固有的关键时间依赖性，因此无法全局传播初始条件约束，并在各种场景下准确捕捉真实解。本文介绍了一种新颖的基于Transformer的框架——PINNsFormer，旨在解决这一局限性。PINNsFormer通过利用多头注意力机制捕捉时间依赖性，能够准确近似PDE解。它将逐点输入转化为伪序列，并用序列损失替代逐点PINNs损失。此外，它引入了一种新颖的激活函数Wavelet，该函数通过深度神经网络实现傅里叶分解的先验。实证结果表明，PINNsFormer在多种场景下（包括PINNs失效模式和高维PDEs）均展现出优越的泛化能力和准确性。此外，PINNsFormer在集成现有PINNs学习方案方面具有灵活性，进一步提升了其性能。