Mean field variational inference (VI) is the problem of finding the closest product (factorized) measure, in the sense of relative entropy, to a given high-dimensional probability measure $\rho$. The well known Coordinate Ascent Variational Inference (CAVI) algorithm aims to approximate this product measure by iteratively optimizing over one coordinate (factor) at a time, which can be done explicitly. Despite its popularity, the convergence of CAVI remains poorly understood. In this paper, we prove the convergence of CAVI for log-concave densities $\rho$. If additionally $\log \rho$ has Lipschitz gradient, we find a linear rate of convergence, and if also $\rho$ is strongly log-concave, we find an exponential rate. Our analysis starts from the observation that mean field VI, while notoriously non-convex in the usual sense, is in fact displacement convex in the sense of optimal transport when $\rho$ is log-concave. This allows us to adapt techniques from the optimization literature on coordinate descent algorithms in Euclidean space.

翻译：平均场变分推理 (VI) 是在相对熵意义下，寻找与给定高维概率测度 ρ 最接近的乘积（因子化）测度的问题。著名的坐标上升变分推理 (CAVI) 算法通过每次迭代优化一个坐标（因子）来近似此乘积测度，该过程可显式执行。尽管该算法广受欢迎，但其收敛性至今仍未被充分理解。本文证明了对数凹密度 ρ 下 CAVI 的收敛性。若 log ρ 进一步具有 Lipschitz 梯度，我们得到线性收敛速率；若 ρ 还是强对数凹的，则得到指数收敛速率。我们的分析始于一个观察：尽管平均场 VI 在通常意义下是著名的非凸问题，但当 ρ 为对数凹时，它在最优传输意义下实际上是位移凸的。这使我们得以借鉴欧氏空间中关于坐标下降算法的优化文献中的技术。