The analytic inference, e.g. predictive distribution being in closed form, may be an appealing benefit for machine learning practitioners when they treat wide neural networks as Gaussian process in Bayesian setting. The realistic widths, however, are finite and cause weak deviation from the Gaussianity under which partial marginalization of random variables in a model is straightforward. On the basis of multivariate Edgeworth expansion, we propose a non-Gaussian distribution in differential form to model a finite set of outputs from a random neural network, and derive the corresponding marginal and conditional properties. Thus, we are able to derive the non-Gaussian posterior distribution in Bayesian regression task. In addition, in the bottlenecked deep neural networks, a weight space representation of deep Gaussian process, the non-Gaussianity is investigated through the marginal kernel.

翻译：在贝叶斯框架下将宽神经网络视为高斯过程时，解析推断（例如预测分布具有闭合形式）可能对机器学习从业者具有显著吸引力。然而实际网络宽度是有限的，这会导致其弱偏离高斯性，而在高斯性假设下模型随机变量的部分边缘化是直接的。基于多元Edgeworth展开，我们提出一种微分形式的非高斯分布来建模随机神经网络的有限输出集合，并推导出相应的边缘与条件性质。由此，我们能够在贝叶斯回归任务中推导出非高斯后验分布。此外，在瓶颈深度神经网络（深层高斯过程的权值空间表示）中，通过边缘核探究了其非高斯性。