Flexible models for probability distributions are an essential ingredient in many machine learning tasks. We develop and investigate a new class of probability distributions, which we call a Squared Neural Family (SNEFY), formed by squaring the 2-norm of a neural network and normalising it with respect to a base measure. Following the reasoning similar to the well established connections between infinitely wide neural networks and Gaussian processes, we show that SNEFYs admit closed form normalising constants in many cases of interest, thereby resulting in flexible yet fully tractable density models. SNEFYs strictly generalise classical exponential families, are closed under conditioning, and have tractable marginal distributions. Their utility is illustrated on a variety of density estimation, conditional density estimation, and density estimation with missing data tasks.

翻译：灵活的概略分布模型是众多机器学习任务中的基本要素。我们开发并研究了一类新的概率分布，称之为平方神经族（SNEFY），其通过神经网络二范数的平方并相对于基测度进行归一化而构建。遵循与无限宽神经网络和高斯过程之间经典联系相似的推理路径，我们证明SNEFY在许多实际情形下具备闭式归一化常数，从而产生灵活且完全可处理的密度模型。SNEFY严格泛化经典指数族，在条件分布下保持封闭性，并具有可处理的边缘分布。其实际应用价值通过多种密度估计、条件密度估计及含缺失数据密度估计任务得到验证。