We derive upper bounds on the Wasserstein distance ($W_1$), with respect to $\sup$-norm, between any continuous $\mathbb{R}^d$ valued random field indexed by the $n$-sphere and the Gaussian, based on Stein's method. We develop a novel Gaussian smoothing technique that allows us to transfer a bound in a smoother metric to the $W_1$ distance. The smoothing is based on covariance functions constructed using powers of Laplacian operators, designed so that the associated Gaussian process has a tractable Cameron-Martin or Reproducing Kernel Hilbert Space. This feature enables us to move beyond one dimensional interval-based index sets that were previously considered in the literature. Specializing our general result, we obtain the first bounds on the Gaussian random field approximation of wide random neural networks of any depth and Lipschitz activation functions at the random field level. Our bounds are explicitly expressed in terms of the widths of the network and moments of the random weights. We also obtain tighter bounds when the activation function has three bounded derivatives.
翻译:我们利用斯坦因方法,推导了以$\sup$-范数度量的、定义在$n$维球面上的任意连续$\mathbb{R}^d$值随机场与高斯随机场之间的Wasserstein距离($W_1$)的上界。我们提出了一种新颖的高斯平滑技术,能够将较平滑度量下的界限转化为$W_1$距离的界限。该平滑技术基于利用拉普拉斯算子幂次构造的协方差函数,使得相关高斯过程具有易处理的Cameron-Martin空间或再生核希尔伯特空间。这一特性使我们能够突破文献中此前仅限于一维区间索引集的局限。通过将我们的通用结果具体化,我们首次获得了任意深度和Lipschitz激活函数的宽随机神经网络在高斯随机场近似层面的界限。这些界限明确地用网络宽度和随机权重的矩来表示。当激活函数具有三阶有界导数时,我们还得到了更紧的界限。