The analytic inference, e.g. predictive distribution being in closed form, may be an appealing benefit for machine learning practitioners when they treat wide neural networks as Gaussian process in Bayesian setting. The realistic widths, however, are finite and cause weak deviation from the Gaussianity under which partial marginalization of random variables in a model is straightforward. On the basis of multivariate Edgeworth expansion, we propose a non-Gaussian distribution in differential form to model a finite set of outputs from a random neural network, and derive the corresponding marginal and conditional properties. Thus, we are able to derive the non-Gaussian posterior distribution in Bayesian regression task. In addition, in the bottlenecked deep neural networks, a weight space representation of deep Gaussian process, the non-Gaussianity is investigated through the marginal kernel.

翻译：在贝叶斯框架下，将宽神经网络视为高斯过程时，分析性推断（例如预测分布具有闭式表达）可能成为机器学习实践者青睐的优势。然而，实际网络宽度是有限的，这会导致与高斯性的弱偏离——在高斯性条件下，模型中对随机变量的部分边缘化是直接的。基于多元Edgeworth展开，我们提出了一种微分形式的非高斯分布，用于建模随机神经网络输出的有限集合，并推导了相应的边缘与条件性质。由此，我们能够在贝叶斯回归任务中推导出非高斯后验分布。此外，在瓶颈式深度神经网络（即深度高斯过程的权重空间表示）中，通过边缘核研究了非高斯性。