Mode collapse, the failure to capture one or more modes when targetting a multimodal distribution, is a central challenge in modern variational inference. In this work, we provide a mathematical analysis of annealing based strategies for mitigating mode collapse in a tractable setting: learning a Gaussian mixture, where mode collapse is known to arise. Leveraging a low dimensional summary statistics description, we precisely characterize the interplay between the initial temperature and the annealing rate, and derive a sharp formula for the probability of mode collapse. Our analysis shows that an appropriately chosen annealing scheme can robustly prevent mode collapse. Finally, we present numerical evidence that these theoretical tradeoffs qualitatively extend to neural network based models, RealNVP normalizing flows, providing guidance for designing annealing strategies mitigating mode collapse in practical variational inference pipelines.

翻译：模态坍塌，即在针对多峰分布时未能捕捉到一个或多个模态，是现代变分推断中的一个核心挑战。本文在可处理的场景下——学习已知会出现模态坍塌的高斯混合模型，对基于退火的缓解模态坍塌策略进行了数学分析。利用低维汇总统计量的描述，我们精确刻画了初始温度与退火速率之间的相互作用，并推导出模态坍塌概率的精确公式。分析表明，适当选择的退火方案能够稳健地防止模态坍塌。最后，我们提供的数值证据表明，这些理论权衡关系在基于神经网络的模型（RealNVP 归一化流）中具有定性上的可延伸性，从而为在实际变分推断流程中设计缓解模态坍塌的退火策略提供了指导。