We establish universality and expression rate bounds for a class of neural Deep Operator Networks (DON) emulating Lipschitz (or H\"older) continuous maps $\mathcal G:\mathcal X\to\mathcal Y$ between (subsets of) separable Hilbert spaces $\mathcal X$, $\mathcal Y$. The DON architecture considered uses linear encoders $\mathcal E$ and decoders $\mathcal D$ via (biorthogonal) Riesz bases of $\mathcal X$, $\mathcal Y$, and an approximator network of an infinite-dimensional, parametric coordinate map that is Lipschitz continuous on the sequence space $\ell^2(\mathbb N)$. Unlike previous works ([Herrmann, Schwab and Zech: Neural and Spectral operator surrogates: construction and expression rate bounds, SAM Report, 2022], [Marcati and Schwab: Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations, SAM Report, 2022]), which required for example $\mathcal G$ to be holomorphic, the present expression rate results require mere Lipschitz (or H\"older) continuity of $\mathcal G$. Key in the proof of the present expression rate bounds is the use of either super-expressive activations (e.g. [Yarotski: Elementary superexpressive activations, Int. Conf. on ML, 2021], [Shen, Yang and Zhang: Neural network approximation: Three hidden layers are enough, Neural Networks, 2021], and the references there) which are inspired by the Kolmogorov superposition theorem, or of nonstandard NN architectures with standard (ReLU) activations as recently proposed in [Zhang, Shen and Yang: Neural Network Architecture Beyond Width and Depth, Adv. in Neural Inf. Proc. Sys., 2022]. We illustrate the abstract results by approximation rate bounds for emulation of a) solution operators for parametric elliptic variational inequalities, and b) Lipschitz maps of Hilbert-Schmidt operators.

翻译：本文建立了一类神经深度算子网络（DON）对可分离希尔伯特空间$\mathcal X$，$\mathcal Y$（子集）之间的Lipschitz（或Hölder）连续映射$\mathcal G:\mathcal X\to\mathcal Y$进行模拟的普适性和表示速率界。所考虑的DON架构使用线性编码器$\mathcal E$和解码器$\mathcal D$（通过$\mathcal X$、$\mathcal Y$的双正交Riesz基），以及一个无限维参数坐标映射的逼近网络，该映射在序列空间$\ell^2(\mathbb N)$上是Lipschitz连续的。与先前需要$\mathcal G$为全纯映射的工作（[Herrmann, Schwab and Zech: Neural and Spectral operator surrogates: construction and expression rate bounds, SAM Report, 2022]，[Marcati and Schwab: Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations, SAM Report, 2022]）不同，本文的表示速率结果仅要求$\mathcal G$具有Lipschitz（或Hölder）连续性。证明当前表示速率界的关键在于使用超级表达激活函数（例如[Yarotski: Elementary superexpressive activations, Int. Conf. on ML, 2021]，[Shen, Yang and Zhang: Neural network approximation: Three hidden layers are enough, Neural Networks, 2021]及其参考文献），其灵感来源于Kolmogorov叠加定理，或使用近期[Zhang, Shen and Yang: Neural Network Architecture Beyond Width and Depth, Adv. in Neural Inf. Proc. Sys., 2022]提出的具有标准（ReLU）激活函数的非标准神经网络架构。我们通过以下模拟的逼近速率界来阐述抽象结果：a) 参数椭圆变分不等式的解算子，b) Hilbert-Schmidt算子的Lipschitz映射。