Learning partial differential equations' (PDEs) solution operators is an essential problem in machine learning. However, there are several challenges for learning operators in practical applications like the irregular mesh, multiple input functions, and complexity of the PDEs' solution. To address these challenges, we propose a general neural operator transformer (GNOT), a scalable and effective transformer-based framework for learning operators. By designing a novel heterogeneous normalized attention layer, our model is highly flexible to handle multiple input functions and irregular meshes. Besides, we introduce a geometric gating mechanism which could be viewed as a soft domain decomposition to solve the multi-scale problems. The large model capacity of the transformer architecture grants our model the possibility to scale to large datasets and practical problems. We conduct extensive experiments on multiple challenging datasets from different domains and achieve a remarkable improvement compared with alternative methods. Our code and data are publicly available at \url{https://github.com/thu-ml/GNOT}.

翻译：学习偏微分方程的解算子是机器学习中的一个关键问题。然而，在实际应用中，学习算子面临诸多挑战，如不规则网格、多输入函数以及偏微分方程解的复杂性。为应对这些挑战，我们提出了一种通用神经算子Transformer（GNOT），这是一种可扩展且高效的基于Transformer的算子学习框架。通过设计一种新颖的异构归一化注意力层，我们的模型能够灵活处理多输入函数和不规则网格。此外，我们引入了一种几何门控机制，可视为一种软域分解方法，用于解决多尺度问题。Transformer架构的大模型容量使我们的模型能够扩展到大规模数据集和实际问题中。我们在来自不同领域的多个具有挑战性的数据集上进行了广泛实验，与替代方法相比，取得了显著改进。我们的代码和数据已在 \url{https://github.com/thu-ml/GNOT} 公开提供。