Variational inference (VI) has emerged as a popular method for approximate inference for high-dimensional Bayesian models. In this paper, we propose a novel VI method that extends the naive mean field via entropic regularization, referred to as $\Xi$-variational inference ($\Xi$-VI). $\Xi$-VI has a close connection to the entropic optimal transport problem and benefits from the computationally efficient Sinkhorn algorithm. We show that $\Xi$-variational posteriors effectively recover the true posterior dependency, where the dependence is downweighted by the regularization parameter. We analyze the role of dimensionality of the parameter space on the accuracy of $\Xi$-variational approximation and how it affects computational considerations, providing a rough characterization of the statistical-computational trade-off in $\Xi$-VI. We also investigate the frequentist properties of $\Xi$-VI and establish results on consistency, asymptotic normality, high-dimensional asymptotics, and algorithmic stability. We provide sufficient criteria for achieving polynomial-time approximate inference using the method. Finally, we demonstrate the practical advantage of $\Xi$-VI over mean-field variational inference on simulated and real data.

翻译：变分推理（VI）已成为高维贝叶斯模型中近似推理的流行方法。本文提出一种全新的VI方法，通过熵正则化扩展朴素平均场，称为$\Xi$-变分推理（$\Xi$-VI）。$\Xi$-VI与熵最优传输问题密切相关，并受益于计算高效的Sinkhorn算法。我们证明$\Xi$-变分后验能有效恢复真实后验依赖关系，其中依赖强度由正则化参数削弱。我们分析了参数空间维度对$\Xi$-变分近似精度的影响及其对计算复杂度的作用，初步刻画了$\Xi$-VI中的统计-计算权衡。我们还探讨了$\Xi$-VI的频率学派性质，建立了关于一致性、渐近正态性、高维渐近性及算法稳定性的结论。我们给出了利用该方法实现多项式时间近似推理的充分准则。最后，在模拟数据和真实数据上展示了$\Xi$-VI相较于平均场变分推理的实际优势。