There is a growing interest on large-width asymptotic properties of Gaussian neural networks (NNs), namely NNs whose weights are initialized according to Gaussian distributions. A well-established result is that, as the width goes to infinity, a Gaussian NN converges in distribution to a Gaussian stochastic process, which provides an asymptotic or qualitative Gaussian approximation of the NN. In this paper, we introduce some non-asymptotic or quantitative Gaussian approximations of Gaussian NNs, quantifying the approximation error with respect to some popular distances for (probability) distributions, e.g. the $1$-Wasserstein distance, the total variation distance and the Kolmogorov-Smirnov distance. Our results rely on the use of second-order Gaussian Poincar\'e inequalities, which provide tight estimates of the approximation error, with optimal rates. This is a novel application of second-order Gaussian Poincar\'e inequalities, which are well-known in the probabilistic literature for being a powerful tool to obtain Gaussian approximations of general functionals of Gaussian stochastic processes. A generalization of our results to deep Gaussian NNs is discussed.

翻译：随着高斯神经网络（即权重根据高斯分布初始化的神经网络）的大宽度渐近性质研究日益受到关注。一个已充分确立的结论是：当网络宽度趋于无穷时，高斯神经网络在分布上收敛于高斯随机过程，这为神经网络提供了渐近或定性的高斯逼近。本文引入高斯神经网络的一些非渐近或定量高斯逼近方法，针对（概率）分布领域若干常用距离（如$1$-Wasserstein距离、全变差距离和Kolmogorov-Smirnov距离）量化逼近误差。我们的结果基于二阶高斯庞加莱不等式，该不等式能够提供具有最优速率的逼近误差紧致估计。这是二阶高斯庞加莱不等式的新颖应用——该不等式在概率论文献中作为获得高斯随机过程一般泛函的高斯逼近的强大工具而闻名。本文进一步讨论了将结果推广至深度高斯神经网络的方案。