Data-driven learning of partial differential equations' solution operators has recently emerged as a promising paradigm for approximating the underlying solutions. The solution operators are usually parameterized by deep learning models that are built upon problem-specific inductive biases. An example is a convolutional or a graph neural network that exploits the local grid structure where functions' values are sampled. The attention mechanism, on the other hand, provides a flexible way to implicitly exploit the patterns within inputs, and furthermore, relationship between arbitrary query locations and inputs. In this work, we present an attention-based framework for data-driven operator learning, which we term Operator Transformer (OFormer). Our framework is built upon self-attention, cross-attention, and a set of point-wise multilayer perceptrons (MLPs), and thus it makes few assumptions on the sampling pattern of the input function or query locations. We show that the proposed framework is competitive on standard benchmark problems and can flexibly be adapted to randomly sampled input.

翻译：数据驱动的偏微分方程解算子学习近期已成为近似求解方程的一种有前景的新范式。解算子通常由基于问题特定归纳偏置构建的深度学习模型参数化，例如利用函数值采样点局部网格结构的卷积神经网络或图神经网络。而注意力机制则提供了一种灵活的方式，既能隐式利用输入中的模式，又能捕捉任意查询位置与输入之间的关联。本研究提出一种基于注意力的数据驱动算子学习框架——算子Transformer（OFormer）。该框架由自注意力、交叉注意力及一组逐点多层感知机（MLPs）构成，因此对输入函数的采样模式或查询位置几乎没有预设假设。实验表明，该框架在标准基准问题上具有竞争力，并能灵活适配随机采样的输入。