In this paper, we study random neural networks which are single-hidden-layer feedforward neural networks whose weights and biases are randomly initialized. After this random initialization, only the linear readout needs to be trained, which can be performed efficiently, e.g., by the least squares method. By viewing random neural networks as Banach space-valued random variables, we prove their universal approximation properties within suitable Bochner spaces. Hereby, the corresponding Banach space can be more general than the space of continuous functions over a compact subset of a Euclidean space, namely, e.g., an $L^p$-space or a Sobolev space, where the latter includes the approximation of the derivatives. Moreover, we derive some approximation rates and develop an explicit algorithm to learn a deterministic function by a random neural network. In addition, we provide a full error analysis and study when random neural networks overcome the curse of dimensionality in the sense that the training costs scale at most polynomially in the input and output dimension. Furthermore, we show in two numerical examples the empirical advantages of random neural networks compared to fully trained deterministic neural networks.

翻译：本文研究随机神经网络，即权重和偏置随机初始化的单隐层前馈神经网络。随机初始化后，仅需训练线性读出层，这可以通过最小二乘法等方法高效完成。通过将随机神经网络视为Banach空间值随机变量，我们证明了它们在合适Bochner空间内的普适逼近性质。此处的Banach空间比欧氏空间紧子集上的连续函数空间更一般，例如$L^p$空间或Sobolev空间（后者包含导数的逼近）。此外，我们推导了一些逼近速率，并开发了一种通过随机神经网络学习确定性函数的显式算法。同时，我们提供完整的误差分析，并研究随机神经网络何时能克服维数灾难——即训练成本随输入与输出维度最多呈多项式增长。最后，通过两个数值示例展示了随机神经网络相较于完全训练确定性神经网络的实证优势。