We generalize the universal approximation theorem for functional input neural networks (FNN) to differentiable maps by including the approximation of the derivatives. A FNN maps the input from a possibly infinite-dimensional weighted manifold to the real-valued hidden layer, on which a non-linear scalar activation function is applied, and then returns the output into a Banach space via some linear readouts. By proving a weighted Nachbin theorem, we establish a universal approximation theorem (UAT) for differentiable maps, which goes beyond the usual formulation on compact sets and also includes the approximation of the derivatives. This leads us to approximation results for non-anticipative functionals including the horizontal and vertical derivatives. As a further application, we show that linear functions of the signature are able to approximate path space functionals including their directional derivatives.

翻译：我们将函数输入神经网络（FNN）的万能逼近定理推广至可微映射，包括对导数的逼近。FNN将可能来自无限维加权流形的输入映射至实值隐层，在该隐层上施加非线性标量激活函数，随后通过某些线性读出将结果输出至Banach空间。通过证明加权Nachbin定理，我们建立了可微映射的万能逼近定理（UAT），该定理超越了紧集上的常规表述，并涵盖了对导数的逼近。这进而导出了非适应泛函（包括水平与垂直导数）的逼近结果。进一步的应用表明，签名线性函数能够逼近包含方向导数的路径空间泛函。