We introduce a highly expressive yet distinctly tractable family for black-box variational inference (BBVI). Each member of this family is a weighted product of experts (PoE), and each weighted expert in the product is proportional to a multivariate $t$-distribution. These products of experts can model distributions with skew, heavy tails, and multiple modes, but to use them for BBVI, we must be able to sample from their densities. We show how to do this by reformulating these products of experts as latent variable models with auxiliary Dirichlet random variables. These Dirichlet variables emerge from a Feynman identity, originally developed for loop integrals in quantum field theory, that expresses the product of multiple fractions (or in our case, $t$-distributions) as an integral over the simplex. We leverage this simplicial latent space to draw weighted samples from these products of experts -- samples which BBVI then uses to find the PoE that best approximates a target density. Given a collection of experts, we derive an iterative procedure to optimize the exponents that determine their geometric weighting in the PoE. At each iteration, this procedure minimizes a regularized Fisher divergence to match the scores of the variational and target densities at a batch of samples drawn from the current approximation. This minimization reduces to a convex quadratic program, and we prove under general conditions that these updates converge exponentially fast to a near-optimal weighting of experts. We conclude by evaluating this approach on a variety of synthetic and real-world target distributions.

翻译：我们为黑盒变分推断引入了一个高度表达性且具有独特可处理性的分布族。该族中的每个成员均为加权专家乘积，且乘积中的每个加权专家与多元$t$分布成比例。这些专家乘积能够建模具有偏态、重尾和多峰特性的分布，但若将其用于黑盒变分推断，我们必须能够从其密度函数中采样。我们通过将这些专家乘积重新表述为带有辅助狄利克雷随机变量的潜变量模型，展示了如何实现采样。这些狄利克雷变量源于一个最初为量子场论中环路积分开发的费曼恒等式，该恒等式将多个分式（在我们的案例中即$t$分布）的乘积表示为单纯形上的积分。我们利用这个单纯形潜空间从这些专家乘积中抽取加权样本——黑盒变分推断随后使用这些样本来寻找最接近目标密度的专家乘积。给定一组专家，我们推导了一种迭代优化程序来确定其在专家乘积中几何加权的指数。在每次迭代中，该程序通过最小化正则化费希尔散度，使变分密度与目标密度在当前近似分布抽取的一批样本处的得分相匹配。此最小化问题可简化为一个凸二次规划，并且我们在一般条件下证明这些更新以指数速度收敛至接近最优的专家加权方案。最后，我们在多种合成与真实世界目标分布上评估了该方法。