We introduce so-called functional input neural networks defined on a possibly infinite dimensional weighted space with values also in a possibly infinite dimensional output space. To this end, we use an additive family as hidden layer maps and a non-linear activation function applied to each hidden layer. Relying on Stone-Weierstrass theorems on weighted spaces, we can prove a global universal approximation result for generalizations of continuous functions going beyond the usual approximation on compact sets. This then applies in particular to approximation of (non-anticipative) path space functionals via functional input neural networks. As a further application of the weighted Stone-Weierstrass theorem we prove a global universal approximation result for linear functions of the signature. We also introduce the viewpoint of Gaussian process regression in this setting and show that the reproducing kernel Hilbert space of the signature kernels are Cameron-Martin spaces of certain Gaussian processes. This paves the way towards uncertainty quantification for signature kernel regression.

翻译：本文提出了定义在可能无限维加权空间上的所谓函数输入神经网络，其取值同样位于可能无限维的输出空间。为此，我们采用加性族作为隐藏层映射，并对每个隐藏层应用非线性激活函数。基于加权空间上的Stone-Weierstrass定理，我们能够证明一类超越紧集上常规逼近的连续函数推广形式的全局通用逼近定理。该结果特别适用于通过函数输入神经网络对（非适应性）路径空间泛函的逼近。作为加权Stone-Weierstrass定理的进一步应用，我们证明了签名线性函数的全局通用逼近定理。此外，我们引入该框架下的高斯过程回归视角，并证明签名核的再生核希尔伯特空间是特定高斯过程的Cameron-Martin空间。这为签名核回归的不确定性量化奠定了基础。