Neural operators have emerged as a powerful tool for learning the mapping between infinite-dimensional parameter and solution spaces of partial differential equations (PDEs). In this work, we focus on multiscale PDEs that have important applications such as reservoir modeling and turbulence prediction. We demonstrate that for such PDEs, the spectral bias towards low-frequency components presents a significant challenge for existing neural operators. To address this challenge, we propose a hierarchical attention neural operator (HANO) inspired by the hierarchical matrix approach. HANO features a scale-adaptive interaction range and self-attentions over a hierarchy of levels, enabling nested feature computation with controllable linear cost and encoding/decoding of multiscale solution space. We also incorporate an empirical $H^1$ loss function to enhance the learning of high-frequency components. Our numerical experiments demonstrate that HANO outperforms state-of-the-art (SOTA) methods for representative multiscale problems.

翻译：神经算子已成为学习偏微分方程（PDE）无限维参数空间与解空间之间映射的强大工具。本研究聚焦于具有重要应用价值（如油藏建模和湍流预测）的多尺度偏微分方程。我们证明，对于此类偏微分方程，现有神经算子对低频分量的谱偏差构成了重大挑战。为应对这一挑战，我们受层次矩阵方法启发，提出了一种分层注意力神经算子（HANO）。HANO具有尺度自适应的交互范围以及跨层次级别的自注意力机制，能够以可控的线性成本进行嵌套特征计算，并对多尺度解空间进行编码/解码。我们还引入了一个经验性的 $H^1$ 损失函数，以加强对高频分量的学习。数值实验表明，对于代表性的多尺度问题，HANO 的性能优于现有最先进（SOTA）方法。