We introduce a Banach space-valued extension of random feature learning, a data-driven supervised machine learning technique for large-scale kernel approximation. By randomly initializing the feature maps, only the linear readout needs to be trained, which reduces the computational complexity substantially. Viewing random feature models as Banach space-valued random variables, we prove a universal approximation result in the corresponding Bochner space. Moreover, we derive approximation rates and an explicit algorithm to learn an element of the given Banach space by such models. The framework of this paper includes random trigonometric/Fourier regression and in particular random neural networks which are single-hidden-layer feedforward neural networks whose weights and biases are randomly initialized, whence only the linear readout needs to be trained. For the latter, we can then lift the universal approximation property of deterministic neural networks to random neural networks, even within function spaces over non-compact domains, e.g., weighted spaces, $L^p$-spaces, and (weighted) Sobolev spaces, where the latter includes the approximation of the (weak) derivatives. In addition, we analyze when the training costs for approximating a given function grow polynomially in both the input/output dimension and the reciprocal of a pre-specified tolerated approximation error. Furthermore, we demonstrate in a numerical example the empirical advantages of random feature models over their deterministic counterparts.

翻译：我们引入了一种Banach空间值的随机特征学习扩展方法，这是一种用于大规模核逼近的数据驱动监督机器学习技术。通过随机初始化特征映射，仅需训练线性读出层，从而大幅降低计算复杂度。将随机特征模型视为Banach空间值随机变量，我们证明了在相应Bochner空间中的通用逼近结果。此外，我们推导了逼近速率及一种显式算法，用于通过此类模型学习给定Banach空间的元素。本文的框架包括随机三角/傅里叶回归，特别地包含随机神经网络——这是一种权重和偏置被随机初始化的单隐层前馈神经网络，因此仅需训练线性读出层。对于后者，我们可将确定性神经网络的通用逼近性质提升至随机神经网络，甚至在非紧致域上的函数空间（如加权空间、$L^p$空间及（加权）Sobolev空间）中仍然成立，其中Sobolev空间还包含（弱）导数的逼近。此外，我们分析了当输入/输出维度和预设容许逼近误差的倒数以多项式增长时，逼近给定函数的训练成本。最后，通过数值算例展示了随机特征模型相较于其确定性对应模型的实证优势。