We consider the infinite-width limit of a fully connected deep neural network with general weights, and we prove quantitative general bounds on the $2$-Wasserstein distance between the network and its infinite-width Gaussian limit, under appropriate regularity assumptions on the activation function. Our main tool is a Lindeberg principle for Deep Neural Networks, which we use to successively replace the weights on each layer by Gaussian random variables.
翻译:本文研究具有一般权重全连接深度神经网络的无穷宽极限,在激活函数满足适当正则性假设的条件下,我们证明了网络与其无穷宽高斯极限之间的2-瓦瑟斯坦距离的定量一般界。我们的主要工具是深度神经网络的林德伯格原理,通过该原理可逐层将各层权重依次替换为高斯随机变量。