In recent years, functional neural networks have been proposed and studied in order to approximate nonlinear continuous functionals defined on $L^p([-1, 1]^s)$ for integers $s\ge1$ and $1\le p<\infty$. However, their theoretical properties are largely unknown beyond universality of approximation or the existing analysis does not apply to the rectified linear unit (ReLU) activation function. To fill in this void, we investigate here the approximation power of functional deep neural networks associated with the ReLU activation function by constructing a continuous piecewise linear interpolation under a simple triangulation. In addition, we establish rates of approximation of the proposed functional deep ReLU networks under mild regularity conditions. Finally, our study may also shed some light on the understanding of functional data learning algorithms.

翻译：近年来，为了逼近定义在$L^p([-1, 1]^s)$（其中整数$s\ge1$，$1\le p<\infty$）上的非线性连续泛函，研究者提出并研究了函数型神经网络。然而，除普适逼近性质外，这些网络的理论特性在很大程度上尚属未知，且现有分析不适用于修正线性单元（ReLU）激活函数。为填补这一空白，我们通过简单三角剖分下的连续分段线性插值，研究了与ReLU激活函数相关的函数型深度神经网络的逼近能力。此外，在温和的正则性条件下，我们建立了所提出的函数型深度ReLU网络的逼近速率。最后，我们的研究也可能为理解函数型数据学习算法提供一些启示。