Learning partial differential equations' (PDEs) solution operators is an essential problem in machine learning. However, there are several challenges for learning operators in practical applications like the irregular mesh, multiple input functions, and complexity of the PDEs' solution. To address these challenges, we propose a general neural operator transformer (GNOT), a scalable and effective transformer-based framework for learning operators. By designing a novel heterogeneous normalized attention layer, our model is highly flexible to handle multiple input functions and irregular mesh. Besides, we introduce a geometric gating mechanism which could be viewed as a soft domain decomposition to solve the multi-scale problems. The large model capacity of transformer architecture grants our model the possibility to scale to large datasets and practical problems. We conduct extensive experiments on multiple challenging datasets from different domains and achieve a remarkable improvement compared with alternative methods.

翻译：学习偏微分方程（PDE）的解算子是机器学习中的一个核心问题。然而，在实际应用中，学习算子面临诸多挑战，如不规则网格、多输入函数以及PDE解的复杂性。为解决这些问题，我们提出了一种通用神经算子Transformer（GNOT），这是一种基于Transformer的可扩展且高效的算子学习框架。通过设计一种新型的异构归一化注意力层，我们的模型能够灵活处理多输入函数和不规则网格。此外，我们引入了几何门控机制，可视为一种软区域分解方法，用于解决多尺度问题。Transformer架构的大模型容量使我们的模型能够扩展到大规模数据集和实际问题。我们在多个来自不同领域的具有挑战性的数据集上进行了广泛实验，与替代方法相比取得了显著改进。