In this paper, we study random neural networks which are single-hidden-layer feedforward neural networks whose weights and biases are randomly initialized. After this random initialization, only the linear readout needs to be trained, which can be performed efficiently, e.g., by the least squares method. By viewing random neural networks as Banach space-valued random variables, we prove a universal approximation theorem within a large class of Bochner spaces. Hereby, the corresponding Banach space can be significantly more general than the space of continuous functions over a compact subset of a Euclidean space, namely, e.g., an $L^p$-space or a Sobolev space, where the latter includes the approximation of the derivatives. Moreover, we derive approximation rates and an explicit algorithm to learn a deterministic function by a random neural network. In addition, we provide a full error analysis and study when random neural networks overcome the curse of dimensionality in the sense that the training costs scale at most polynomially in the input and output dimension. Furthermore, we show in two numerical examples the empirical advantages of random neural networks compared to fully trained deterministic neural networks.

翻译：本文研究随机神经网络，即权重和偏置随机初始化的单隐层前馈神经网络。随机初始化后，仅需训练线性读出层，可通过最小二乘法等方法高效实现。通过将随机神经网络视为巴拿赫空间值随机变量，我们在广泛的博赫纳空间类中证明了通用逼近定理。其中，相应的巴拿赫空间可显著比欧几里得空间紧子集上的连续函数空间更泛化，例如$L^p$空间或索博列夫空间，后者包含导数的逼近。此外，我们推导了逼近速率及利用随机神经网络学习确定性函数的显式算法。我们还提供了完整的误差分析，并研究了随机神经网络在输入输出维度的训练成本至多为多项式规模时如何克服维数灾难。最后，通过两个数值示例展示了随机神经网络相对于完全训练确定性神经网络的实证优势。