A Random Vector Functional Link (RVFL) network is a depth-2 neural network with random inner weights and biases. As only the outer weights of such architectures need to be learned, the learning process boils down to a linear optimization task, allowing one to sidestep the pitfalls of nonconvex optimization problems. In this paper, we prove that an RVFL with ReLU activation functions can approximate Lipschitz continuous functions provided its hidden layer is exponentially wide in the input dimension. Although it has been established before that such approximation can be achieved in $L_2$ sense, we prove it for $L_\infty$ approximation error and Gaussian inner weights. To the best of our knowledge, our result is the first of this kind. We give a nonasymptotic lower bound for the number of hidden layer nodes, depending on, among other things, the Lipschitz constant of the target function, the desired accuracy, and the input dimension. Our method of proof is rooted in probability theory and harmonic analysis.

翻译：随机向量函数链接（RVFL）网络是一种深度为2的神经网络，其内部权重和偏置为随机值。由于该架构仅需学习外部权重，学习过程简化为线性优化任务，从而避免了非凸优化问题的陷阱。本文证明，若RVFL网络采用ReLU激活函数，且隐藏层宽度随输入维度呈指数增长，则该网络可逼近Lipschitz连续函数。尽管已有研究表明此类逼近可在$L_2$意义下实现，但我们针对$L_\infty$逼近误差及高斯内部权重给出了证明。据我们所知，本文结果是该领域的首次突破。我们给出了隐藏层节点数的非渐近下界，其依赖目标函数的Lipschitz常数、期望精度以及输入维度等因素。我们的证明方法植根于概率论与调和分析。