Learning partial differential equations' (PDEs) solution operators is an essential problem in machine learning. However, there are several challenges for learning operators in practical applications like the irregular mesh, multiple input functions, and complexity of the PDEs' solution. To address these challenges, we propose a general neural operator transformer (GNOT), a scalable and effective transformer-based framework for learning operators. By designing a novel heterogeneous normalized attention layer, our model is highly flexible to handle multiple input functions and irregular meshes. Besides, we introduce a geometric gating mechanism which could be viewed as a soft domain decomposition to solve the multi-scale problems. The large model capacity of the transformer architecture grants our model the possibility to scale to large datasets and practical problems. We conduct extensive experiments on multiple challenging datasets from different domains and achieve a remarkable improvement compared with alternative methods. Our code and data are publicly available at \url{https://github.com/thu-ml/GNOT}.

翻译：学习偏微分方程的解算子问题是机器学习中的一个基本问题。然而，在实际应用中，算子学习面临若干挑战，例如不规则网格、多输入函数以及偏微分方程解的复杂性。为解决这些问题，我们提出了一种通用神经算子Transformer（GNOT），这是一种可扩展且高效的基于Transformer的算子学习框架。通过设计新颖的异构归一化注意力层，我们的模型具有高度灵活性，能够处理多输入函数和不规则网格。此外，我们引入了一种几何门控机制，可视为一种软区域分解方法，用于解决多尺度问题。Transformer架构的大模型容量使我们的模型具备扩展到大规模数据集和实际问题的可能性。我们在来自不同领域的多个具有挑战性的数据集上进行了广泛实验，与替代方法相比取得了显著改进。我们的代码和数据已在\url{https://github.com/thu-ml/GNOT}公开。