While generalized linear mixed models (GLMMs) are a fundamental tool in applied statistics, many specifications -- such as those involving categorical factors with many levels or interaction terms -- can be computationally challenging to estimate due to the need to compute or approximate high-dimensional integrals. Variational inference (VI) methods are a popular way to perform such computations, especially in the Bayesian context. However, naive VI methods can provide unreliable uncertainty quantification. We show that this is indeed the case in the GLMM context, proving that standard VI (i.e. mean-field) dramatically underestimates posterior uncertainty in high-dimensions. We then show how appropriately relaxing the mean-field assumption leads to VI methods whose uncertainty quantification does not deteriorate in high-dimensions, and whose total computational cost scales linearly with the number of parameters and observations. Our theoretical and numerical results focus on GLMMs with Gaussian or binomial likelihoods, and rely on connections to random graph theory to obtain sharp high-dimensional asymptotic analysis. We also provide generic results, which are of independent interest, relating the accuracy of variational inference to the convergence rate of the corresponding coordinate ascent variational inference (CAVI) algorithm for Gaussian targets. Our proposed partially-factorized VI (PF-VI) methodology for GLMMs is implemented in the R package vglmer, see https://github.com/mgoplerud/vglmer . Numerical results with simulated and real data examples illustrate the favourable computation cost versus accuracy trade-off of PF-VI.

翻译：尽管广义线性混合模型（GLMMs）是应用统计学中的基础工具，但许多设定——例如涉及多水平分类因子或交互项的情况——由于需要计算或近似高维积分，其估计过程在计算上极具挑战性。变分推断（VI）方法是执行这类计算的常用手段，尤其在贝叶斯框架中。然而，朴素变分推断方法可能提供不可靠的不确定性量化。我们证明这在GLMM背景下确实成立，并指出标准变分推断（即均值场）会严重低估高维参数的后验不确定性。随后我们展示如何适当放松均值场假设，从而得到一种变分推断方法，其不确定性量化在高维条件下不会退化，且总计算成本与参数和观测数量呈线性关系。我们的理论与数值结果聚焦于高斯或二项似然的GLMM，并借助随机图理论获得精准的高维渐近分析。我们还提供了独立意义的通用结论，将变分推断的精度与高斯目标对应的坐标上升变分推断（CAVI）算法收敛速率相关联。我们提出的GLMM部分因子化变分推断（PF-VI）方法已在R包vglmer中实现（参见https://github.com/mgoplerud/vglmer ）。基于模拟与真实数据的数值实验结果表明，PF-VI在计算成本与精度权衡方面具有优势。