We introduce a Banach space-valued extension of random feature learning, a data-driven supervised machine learning technique for large-scale kernel approximation. By randomly initializing the feature maps, only the linear readout needs to be trained, which reduces the computational complexity substantially. Viewing random feature models as Banach space-valued random variables, we prove a universal approximation result in the corresponding Bochner space. Moreover, we derive approximation rates and an explicit algorithm to learn an element of the given Banach space by such models. The framework of this paper includes random trigonometric/Fourier regression and in particular random neural networks which are single-hidden-layer feedforward neural networks whose weights and biases are randomly initialized, whence only the linear readout needs to be trained. For the latter, we can then lift the universal approximation property of deterministic neural networks to random neural networks, even within function spaces over non-compact domains, e.g., weighted spaces, $L^p$-spaces, and (weighted) Sobolev spaces, where the latter includes the approximation of the (weak) derivatives. In addition, we analyze when the training costs for approximating a given function grow polynomially in both the input/output dimension and the reciprocal of a pre-specified tolerated approximation error. Furthermore, we demonstrate in a numerical example the empirical advantages of random feature models over their deterministic counterparts.

翻译：本文提出了一种巴拿赫空间值的随机特征学习扩展方法，这是一种用于大规模核逼近的数据驱动监督机器学习技术。通过随机初始化特征映射，仅需训练线性读出层，从而大幅降低计算复杂度。将随机特征模型视为巴拿赫空间值随机变量，我们在相应的博赫纳空间中证明了普适逼近定理。此外，我们推导了逼近速率，并给出显式算法以通过此类模型学习给定巴拿赫空间的元素。本文框架涵盖随机三角/傅里叶回归，特别包含随机神经网络——即权重和偏置被随机初始化的单隐层前馈神经网络，因而仅需训练线性读出层。对于后者，我们可以将确定性神经网络的普适逼近性质推广至随机神经网络，甚至适用于非紧域上的函数空间，例如加权空间、$L^p$空间和（加权）索伯列夫空间，其中后者包含（弱）导数的逼近。此外，我们分析了当逼近给定函数时，训练成本在输入/输出维度与预设容许逼近误差倒数的多项式增长条件。最后，我们通过数值算例展示了随机特征模型相较于确定性对应方法的实证优势。