Variational inference (VI) has emerged as a popular method for approximate inference for high-dimensional Bayesian models. In this paper, we propose a novel VI method that extends the naive mean field via entropic regularization, referred to as $\Xi$-variational inference ($\Xi$-VI). $\Xi$-VI has a close connection to the entropic optimal transport problem and benefits from the computationally efficient Sinkhorn algorithm. We show that $\Xi$-variational posteriors effectively recover the true posterior dependency, where the dependence is downweighted by the regularization parameter. We analyze the role of dimensionality of the parameter space on the accuracy of $\Xi$-variational approximation and how it affects computational considerations, providing a rough characterization of the statistical-computational trade-off in $\Xi$-VI. We also investigate the frequentist properties of $\Xi$-VI and establish results on consistency, asymptotic normality, high-dimensional asymptotics, and algorithmic stability. We provide sufficient criteria for achieving polynomial-time approximate inference using the method. Finally, we demonstrate the practical advantage of $\Xi$-VI over mean-field variational inference on simulated and real data.

翻译：变分推理（VI）已成为高维贝叶斯模型中近似推理的流行方法。本文提出了一种新的变分推理方法，通过熵正则化扩展了朴素平均场方法，称为$\Xi$-变分推理（$\Xi$-VI）。$\Xi$-VI与熵最优输运问题密切相关，并受益于计算高效的Sinkhorn算法。我们证明$\Xi$-变分后验能有效恢复真实后验的依赖关系，其中依赖程度由正则化参数降权。我们分析了参数空间维度对$\Xi$-变分近似精度的影响及其对计算效率的考量，初步刻画了$\Xi$-VI中的统计-计算权衡。此外，我们还研究了$\Xi$-VI的频率学性质，建立了关于一致性、渐近正态性、高维渐近性和算法稳定性的结果。我们提供了使用该方法实现多项式时间近似推理的充分条件。最后，在模拟数据和真实数据上展示了$\Xi$-VI相比平均场变分推理的实际优势。