Simulating physical systems using Partial Differential Equations (PDEs) has become an indispensible part of modern industrial process optimization. Traditionally, numerical solvers have been used to solve the associated PDEs, however recently Transform-based Neural Operators such as the Fourier Neural Operator and Wavelet Neural Operator have received a lot of attention for their potential to provide fast solutions for systems of PDEs. In this work, we investigate the importance of the transform layers to the reported success of transform based neural operators. In particular, we record the cost in terms of performance, if all the transform layers are replaced by learnable linear layers. Surprisingly, we observe that linear layers suffice to provide performance comparable to the best-known transform-based layers and seem to do so with a compute time advantage as well. We believe that this observation can have significant implications for future work on Neural Operators, and might point to other sources of efficiencies for these architectures.

翻译：使用偏微分方程模拟物理系统已成为现代工业过程优化中不可或缺的一部分。传统上，数值求解器被用于求解相关偏微分方程，但近年来，基于变换的神经算子（如傅里叶神经算子和小波神经算子）因其在快速求解偏微分方程系统方面的潜力而备受关注。本研究探讨了变换层对基于变换的神经算子成功表现的重要性。具体而言，我们记录了将所有变换层替换为可学习线性层时的性能代价。令人惊讶的是，我们发现线性层足以提供与已知最优变换层相当的性能，并且似乎具有计算时间优势。我们认为这一发现可能对神经算子的未来研究产生重要影响，并可能指向此类架构的其他效率来源。