We propose stepwise variational inference (VI) with vine copulas: a universal VI procedure that combines vine copulas with a novel stepwise estimation procedure of the variational parameters. Vine copulas consist of a nested sequence of trees built from copulas, where more complex latent dependence can be modeled with increasing number of trees. We propose to estimate the vine copula approximate posterior in a stepwise fashion, tree by tree along the vine structure. Further, we show that the usual backward Kullback-Leibler divergence cannot recover the correct parameters in the vine copula model, thus the evidence lower bound is defined based on the Rényi divergence. Finally, an intuitive stopping criterion for adding further trees to the vine eliminates the need to pre-define a complexity parameter of the variational distribution, as required for most other approaches. Thus, our method interpolates between mean-field VI (MFVI) and full latent dependence. In many applications, in particular sparse Gaussian processes, our method is parsimonious with parameters, while outperforming MFVI.

翻译：我们提出了一种基于藤蔓Copula的分步变分推断方法：这是一种通用变分推断流程，将藤蔓Copula与新颖的变分参数分步估计步骤相结合。藤蔓Copula由基于Copula构建的嵌套树序列构成，当树的数量增加时，可对更复杂的潜在依赖关系进行建模。我们提出沿着藤蔓结构，逐棵树地对藤蔓Copula近似后验进行分步估计。进一步，我们证明标准的后向KL散度无法在藤蔓Copula模型中恢复正确参数，因此证据下界基于Rényi散度来定义。最后，引入一个直观的停止准则来终止向藤蔓中添加更多树的操作，从而无需像大多数其他方法那样预先定义变分分布的复杂度参数。因此，我们的方法在平均场变分推断与全潜在依赖关系之间进行了插值。在许多应用（尤其是稀疏高斯过程）中，我们的方法在参数上更为精简，同时性能优于平均场变分推断。