Mean-field variational inference (MFVI) is a widely used method for approximating high-dimensional probability distributions by product measures. It has been empirically observed that MFVI optimizers often suffer from mode collapse. Specifically, when the target measure $\pi$ is a mixture $\pi = w P_0 + (1 - w) P_1$, the MFVI optimizer tends to place most of its mass near a single component of the mixture. This work provides the first theoretical explanation of mode collapse in MFVI. We introduce the notion to capture the separatedness of the two mixture components -- called $\varepsilon$-separateness -- and derive explicit bounds on the fraction of mass that any MFVI optimizer assigns to each component when $P_0$ and $P_1$ are $\varepsilon$-separated for sufficiently small $\varepsilon$. Our results suggest that the occurrence of mode collapse crucially depends on the relative position of the components. To address this issue, we propose the rotational variational inference (RoVI), which augments MFVI with a rotation matrix. The numerical studies support our theoretical findings and demonstrate the benefits of RoVI.

翻译：均值场变分推断（MFVI）是一种通过乘积测度逼近高维概率分布的常用方法。经验观察表明，MFVI优化器常受模式崩溃问题困扰。具体而言，当目标测度 $\pi$ 为混合分布 $\pi = w P_0 + (1 - w) P_1$ 时，MFVI优化器倾向于将大部分质量集中于混合分布的单一成分附近。本研究首次从理论角度解释了MFVI中的模式崩溃现象。我们引入了一个用于刻画两个混合成分分离程度的概念——称为 $\varepsilon$ 可分离性，并在 $P_0$ 与 $P_1$ 满足充分小 $\varepsilon$ 的 $\varepsilon$ 可分离条件时，推导出任意MFVI优化器分配给各成分质量比例的显式界。研究结果表明，模式崩溃的发生关键取决于各成分的相对位置关系。针对此问题，我们提出了旋转变分推断（RoVI），该方法通过引入旋转矩阵对MFVI进行扩展。数值研究验证了我们的理论发现，并证明了RoVI的优越性。