We highlight a striking difference in behavior between two widely used variants of coordinate ascent variational inference: the sequential and parallel algorithms. While such differences were known in the numerical analysis literature in simpler settings, they remain largely unexplored in the optimization-focused literature on variational inference in more complex models. Focusing on the moderately high-dimensional linear regression problem, we show that the sequential algorithm, although typically slower, enjoys convergence guarantees under more relaxed conditions than the parallel variant, which is often employed to facilitate block-wise updates and improve computational efficiency.

翻译：我们揭示了两种广泛使用的坐标上升变分推断变体——序贯算法与并行算法——在行为上的显著差异。尽管在数值分析的文献中，这些差异在更简单的设置下已为人所知，但在面向优化的变分推断文献中，针对更复杂模型的研究仍较为缺乏。聚焦于中等高维线性回归问题，我们证明序贯算法尽管通常速度较慢，但其收敛性保障所需的条件比并行变体更为宽松；而并行变体则常被用于促进分块更新并提升计算效率。