We introduce so-called functional input neural networks defined on a possibly infinite dimensional weighted space with values also in a possibly infinite dimensional output space. To this end, we use an additive family to map the input weighted space to the hidden layer, on which a non-linear scalar activation function is applied to each neuron, and finally return the output via some linear readouts. Relying on Stone-Weierstrass theorems on weighted spaces, we can prove a global universal approximation result on weighted spaces for continuous functions going beyond the usual approximation on compact sets. This then applies in particular to approximation of (non-anticipative) path space functionals via functional input neural networks. As a further application of the weighted Stone-Weierstrass theorem we prove a global universal approximation result for linear functions of the signature. We also introduce the viewpoint of Gaussian process regression in this setting and emphasize that the reproducing kernel Hilbert space of the signature kernels are Cameron-Martin spaces of certain Gaussian processes. This paves a way towards uncertainty quantification for signature kernel regression.

翻译：我们提出了所谓的函数输入神经网络，其定义在可能无限维的加权空间上，输出也位于可能无限维的输出空间中。为此，我们使用加法族将输入加权空间映射到隐藏层，在该层中，对每个神经元应用非线性标量激活函数，最终通过线性读出层返回输出。基于加权空间上的Stone-Weierstrass定理，我们能够证明加权空间上连续函数的全局通用逼近结果，该结果超越了对紧集上的通常逼近。这一结果特别适用于通过函数输入神经网络对（非预期的）路径空间泛函进行逼近。作为加权Stone-Weierstrass定理的进一步应用，我们证明了签名线性函数的全局通用逼近结果。我们还引入了高斯过程回归在此设置下的视角，并强调签名核的再生核希尔伯特空间是某些高斯过程的Cameron-Martin空间。这为签名核回归的不确定性量化铺平了道路。