We study mean-field variational inference in a Bayesian linear model when the sample size n is comparable to the dimension p. In high dimensions, the common approach of minimizing a Kullback-Leibler divergence from the posterior distribution, or maximizing an evidence lower bound, may deviate from the true posterior mean and underestimate posterior uncertainty. We study instead minimization of the TAP free energy, showing in a high-dimensional asymptotic framework that it has a local minimizer which provides a consistent estimate of the posterior marginals and may be used for correctly calibrated posterior inference. Geometrically, we show that the landscape of the TAP free energy is strongly convex in an extensive neighborhood of this local minimizer, which under certain general conditions can be found by an Approximate Message Passing (AMP) algorithm. We then exhibit an efficient algorithm that linearly converges to the minimizer within this local neighborhood. In settings where it is conjectured that no efficient algorithm can find this local neighborhood, we prove analogous geometric properties for a local minimizer of the TAP free energy reachable by AMP, and show that posterior inference based on this minimizer remains correctly calibrated.

翻译：我们研究了在样本量n与维度p相当时贝叶斯线性模型中的平均场变分推断。在高维情形下，常用的最小化后验分布的Kullback-Leibler散度或最大化证据下界的方法可能会偏离真实后验均值并低估后验不确定性。我们转而研究TAP自由能的最小化，在高维渐近框架下证明其存在一个局部极小元，该极小元能够给出后验边际分布的一致估计，并可用于正确校准的后验推断。在几何上，我们证明TAP自由能在该局部极小元的一个广延邻域内是强凸的，且在某些一般条件下可通过近似消息传递（AMP）算法找到该邻域。随后我们提出一种高效算法，在该局部邻域内线性收敛至极小元。在那些推测无高效算法能到达该局部邻域的情形中，我们证明了AMP可达的TAP自由能局部极小元具有类似的几何性质，并表明基于该极小元的后验推断仍可保持正确校准。