We derive upper bounds on the Wasserstein distance ($W_1$), with respect to $\sup$-norm, between any continuous $\mathbb{R}^d$ valued random field indexed by the $n$-sphere and the Gaussian, based on Stein's method. We develop a novel Gaussian smoothing technique that allows us to transfer a bound in a smoother metric to the $W_1$ distance. The smoothing is based on covariance functions constructed using powers of Laplacian operators, designed so that the associated Gaussian process has a tractable Cameron-Martin or Reproducing Kernel Hilbert Space. This feature enables us to move beyond one dimensional interval-based index sets that were previously considered in the literature. Specializing our general result, we obtain the first bounds on the Gaussian random field approximation of wide random neural networks of any depth and Lipschitz activation functions at the random field level. Our bounds are explicitly expressed in terms of the widths of the network and moments of the random weights. We also obtain tighter bounds when the activation function has three bounded derivatives.
翻译:我们基于Stein方法,推导了以$\sup$-范数度量的、定义在$n$维球面上的任意连续$\mathbb{R}^d$值随机场与高斯随机场之间的Wasserstein距离($W_1$)的上界。我们提出了一种新颖的高斯平滑技术,可将更平滑度量下的界转移到$W_1$距离。该平滑基于使用拉普拉斯算子幂构造的协方差函数,使得关联的高斯过程具有易处理的Cameron-Martin空间或再生核希尔伯特空间。这一特性使我们能够超越文献中此前考虑的基于一维区间的指标集。将我们的通用结果具体化,我们首次在随机场层面获得了任意深度和Lipschitz激活函数的宽随机神经网络的高斯随机场近似界。这些界明确地表示为网络宽度和随机权重矩的函数。当激活函数具有三阶有界导数时,我们还得到了更紧的界。