The limit of infinite width allows for substantial simplifications in the analytical study of overparameterized neural networks. With a suitable random initialization, an extremely large network is well approximated by a Gaussian process, both before and during training. In the present work, we establish a similar result for a simple stochastic architecture whose parameters are random variables. The explicit evaluation of the output distribution allows for a PAC-Bayesian training procedure that directly optimizes the generalization bound. For a large but finite-width network, we show empirically on MNIST that this training approach can outperform standard PAC-Bayesian methods.

翻译：无限宽度的限度可以大大简化对超参数神经网络的分析研究。 有了适当的随机初始化, 一个巨大的网络在培训前后和培训期间都被一个高斯进程所近似。 在目前的工作中, 我们为一个简单的随机变量参数的随机随机随机结构建立类似的结果。 对输出分布的清晰评价允许一个PAC-Bayesian培训程序, 直接优化一般化约束。 对于一个大型但有限的宽度网络, 我们用实验方法向MNIST显示, 这种培训方法可以超过标准的PAC- Bayesian 方法。