The Na\"ive Mean Field (NMF) approximation is widely employed in modern Machine Learning due to the huge computational gains it bestows on the statistician. Despite its popularity in practice, theoretical guarantees for high-dimensional problems are only available under strong structural assumptions (e.g., sparsity). Moreover, existing theory often does not explain empirical observations noted in the existing literature. In this paper, we take a step towards addressing these problems by deriving sharp asymptotic characterizations for the NMF approximation in high-dimensional linear regression. Our results apply to a wide class of natural priors and allow for model mismatch (i.e., the underlying statistical model can be different from the fitted model). We work under an \textit{iid} Gaussian design and the proportional asymptotic regime, where the number of features and the number of observations grow at a proportional rate. As a consequence of our asymptotic characterization, we establish two concrete corollaries: (a) we establish the inaccuracy of the NMF approximation for the log-normalizing constant in this regime, and (b) we provide theoretical results backing the empirical observation that the NMF approximation can be overconfident in terms of uncertainty quantification. Our results utilize recent advances in the theory of Gaussian comparison inequalities. To the best of our knowledge, this is the first application of these ideas to the analysis of Bayesian variational inference problems. Our theoretical results are corroborated by numerical experiments. Lastly, we believe our results can be generalized to non-Gaussian designs and provide empirical evidence to support it.

翻译：朴素平均场（NMF）近似由于为统计学家带来巨大的计算增益，在现代机器学习中被广泛采用。尽管在实践中广受欢迎，但其在高维问题上的理论保证仅能在强结构假设（如稀疏性）下获得。此外，现有理论常无法解释文献中记录的实证观察。本文通过推导高维线性回归中NMF近似的尖锐渐近刻画，为解决这些问题迈出一步。我们的结果适用于一大类自然先验，并允许模型失配（即潜在统计模型可以与拟合模型不同）。我们在独立同分布高斯设计及比例渐近机制下开展工作，其中特征数与观测数以比例速率增长。作为渐近刻画的结果，我们确立了两个具体推论：（a）我们证明了在该机制下NMF近似对于对数正态化常数的不准确性，以及（b）我们提供了支持实证观察的理论结果，即NMF近似在不确定性量化方面可能过于自信。我们的结果利用了高斯比较不等式理论的最新进展。据我们所知，这是这些思想首次应用于贝叶斯变分推断问题的分析。我们的理论结果通过数值实验得到验证。最后，我们相信研究结果可推广至非高斯设计，并提供了支持该观点的实证证据。