Tree-shaped graphical models are widely used for their tractability. However, they unfortunately lack expressive power as they require committing to a particular sparse dependency structure. We propose a novel class of generative models called mixtures of all trees: that is, a mixture over all possible ($n^{n-2}$) tree-shaped graphical models over $n$ variables. We show that it is possible to parameterize this Mixture of All Trees (MoAT) model compactly (using a polynomial-size representation) in a way that allows for tractable likelihood computation and optimization via stochastic gradient descent. Furthermore, by leveraging the tractability of tree-shaped models, we devise fast-converging conditional sampling algorithms for approximate inference, even though our theoretical analysis suggests that exact computation of marginals in the MoAT model is NP-hard. Empirically, MoAT achieves state-of-the-art performance on density estimation benchmarks when compared against powerful probabilistic models including hidden Chow-Liu Trees.

翻译：树形图模型因其可处理性而被广泛使用。然而，它们不幸地缺乏表达能力，因为需要承诺一种特定的稀疏依赖结构。我们提出了一类新的生成模型，称为所有树的混合模型：即，对 $n$ 个变量上所有可能的（$n^{n-2}$ 种）树形图模型的混合。我们证明，可以紧凑地（使用多项式大小的表示）参数化这种所有树的混合（MoAT）模型，从而允许通过随机梯度下降进行可处理的可能性计算和优化。此外，通过利用树形模型的可处理性，我们设计了快速收敛的条件采样算法用于近似推理，尽管我们的理论分析表明在 MoAT 模型中精确计算边际分布是 NP 难的。实验上，在与包括隐式 Chow-Liu 树在内的强大概率模型进行比较时，MoAT 在密度估计基准测试中取得了最先进的性能。