Despite their remarkable success in approximating a wide range of operators defined by PDEs, existing neural operators (NOs) do not necessarily perform well for all physics problems. We focus here on high-frequency waves to highlight possible shortcomings. To resolve these, we propose a subfamily of NOs enabling an enhanced empirical approximation of the nonlinear operator mapping wave speed to solution, or boundary values for the Helmholtz equation on a bounded domain. The latter operator is commonly referred to as the ''forward'' operator in the study of inverse problems. Our methodology draws inspiration from transformers and techniques such as stochastic depth. Our experiments reveal certain surprises in the generalization and the relevance of introducing stochastic depth. Our NOs show superior performance as compared with standard NOs, not only for testing within the training distribution but also for out-of-distribution scenarios. To delve into this observation, we offer an in-depth analysis of the Rademacher complexity associated with our modified models and prove an upper bound tied to their stochastic depth that existing NOs do not satisfy. Furthermore, we obtain a novel out-of-distribution risk bound tailored to Gaussian measures on Banach spaces, again relating stochastic depth with the bound. We conclude by proposing a hypernetwork version of the subfamily of NOs as a surrogate model for the mentioned forward operator.

翻译：尽管现有神经算子在逼近偏微分方程所定义的各种算子方面取得了显著成功，但它们并不一定对所有物理问题都能展现出良好性能。本文聚焦于高频波以揭示其潜在不足。为解决这些问题，我们提出了一类神经算子子集，能够改进对将波速映射至解或有界域上亥姆霍兹方程边界值的非线性算子的经验逼近。后者在反问题研究中通常被称为“正演”算子。我们的方法借鉴了Transformer架构及随机深度等技术。实验揭示了泛化性能的某些意外现象以及引入随机深度的重要性。与标准神经算子相比，我们的算子不仅在训练分布内的测试中表现更优，在分布外场景下也展现出更佳性能。为深入探究这一现象，我们对修改后模型的Rademacher复杂度进行了详尽分析，并证明了与随机深度相关的上界——这是现有神经算子未能满足的。此外，我们针对巴拿赫空间上的高斯测度得到了一个新颖的分布外风险界，再次将随机深度与该界相关联。最后，我们提出将该神经算子子集的超网络版本作为上述正演算子的替代模型。