Probabilistic models based on continuous latent spaces, such as variational autoencoders, can be understood as uncountable mixture models where components depend continuously on the latent code. They have proven to be expressive tools for generative and probabilistic modelling, but are at odds with tractable probabilistic inference, that is, computing marginals and conditionals of the represented probability distribution. Meanwhile, tractable probabilistic models such as probabilistic circuits (PCs) can be understood as hierarchical discrete mixture models, and thus are capable of performing exact inference efficiently but often show subpar performance in comparison to continuous latent-space models. In this paper, we investigate a hybrid approach, namely continuous mixtures of tractable models with a small latent dimension. While these models are analytically intractable, they are well amenable to numerical integration schemes based on a finite set of integration points. With a large enough number of integration points the approximation becomes de-facto exact. Moreover, for a finite set of integration points, the integration method effectively compiles the continuous mixture into a standard PC. In experiments, we show that this simple scheme proves remarkably effective, as PCs learnt this way set new state of the art for tractable models on many standard density estimation benchmarks.

翻译：基于连续潜在空间的概率模型（如变分自编码器）可被理解为一种不可数混合模型，其中各分量连续依赖于潜在编码。此类模型已被证明是生成建模与概率建模中富有表现力的工具，但其难以实现可处理的概率推理，即无法高效计算所表示概率分布的边缘概率与条件概率。另一方面，概率电路等可处理概率模型可被理解为分层离散混合模型，因而能够高效执行精确推理，但其性能通常逊于连续潜在空间模型。本文研究了一种混合方法，即具有小潜在维度的可处理模型的连续混合。尽管此类模型在解析上不可处理，但非常适合基于有限积分点集的数值积分方案。当积分点数量足够大时，该近似实际上等价于精确解。此外，对于有限积分点集，积分方法能有效将连续混合模型编译为标准概率电路。实验表明，这一简单方案具有显著有效性，通过该方法学习的概率电路在许多标准密度估计基准测试中为可处理模型建立了新的最优性能。