We introduce so-called functional input neural networks defined on a possibly infinite dimensional weighted space with values also in a possibly infinite dimensional output space. To this end, we use an additive family as hidden layer maps and a non-linear activation function applied to each hidden layer. Relying on Stone-Weierstrass theorems on weighted spaces, we can prove a global universal approximation result for generalizations of continuous functions going beyond the usual approximation on compact sets. This then applies in particular to approximation of (non-anticipative) path space functionals via functional input neural networks. As a further application of the weighted Stone-Weierstrass theorem we prove a global universal approximation result for linear functions of the signature. We also introduce the viewpoint of Gaussian process regression in this setting and show that the reproducing kernel Hilbert space of the signature kernels are Cameron-Martin spaces of certain Gaussian processes. This paves the way towards uncertainty quantification for signature kernel regression.

翻译：我们引入了定义在可能无限维加权空间上的所谓函数输入神经网络，其输出值也位于可能无限维的输出空间中。为此，我们采用加性族作为隐层映射，并对每个隐层应用非线性激活函数。基于加权空间上的Stone-Weierstrass定理，我们能够证明连续函数推广形式的全局一致逼近结果，该结果超越了通常的紧集上的逼近。这一结果特别适用于通过函数输入神经网络对（非预测型）路径空间泛函的逼近。作为加权Stone-Weierstrass定理的进一步应用，我们证明了签名线性函数的全局一致逼近结果。我们还在此框架下引入了高斯过程回归的视角，并证明了签名核的再生核希尔伯特空间是某些高斯过程的Cameron-Martin空间。这为签名核回归的不确定性量化铺平了道路。