The Laplace approximation is a popular method for providing posterior mean and variance estimates. But can we trust these estimates for practical use? One might consider using rate-of-convergence bounds for the Bayesian Central Limit Theorem (BCLT) to provide quality guarantees for the Laplace approximation. But the bounds in existing versions of the BCLT either: require knowing the true data-generating parameter, are asymptotic in the number of samples, do not control the Bayesian posterior mean, or apply only to narrow classes of models. Our work provides the first closed-form, finite-sample quality bounds for the Laplace approximation that simultaneously (1) do not require knowing the true parameter, (2) control posterior means and variances, and (3) apply generally to models that satisfy the conditions of the asymptotic BCLT. In fact, our bounds work even in the presence of misspecification. We compute exact constants in our bounds for a variety of standard models, including logistic regression, and numerically demonstrate their utility. We provide a framework for analysis of more complex models.
翻译:拉普拉斯近似是一种常用于提供后验均值和方差估计的方法。然而,这些估计在实际应用中是否值得信赖?一种思路是利用贝叶斯中心极限定理的收敛速度边界来为拉普拉斯近似提供质量保证。但现有版本的贝叶斯中心极限定理中的界存在以下局限性:需要知道真实数据生成参数、随样本数量呈渐近性、无法控制贝叶斯后验均值,或仅适用于狭窄的模型类别。本文首次提出了拉普拉斯近似的闭式、有限样本质量界,该界同时满足以下条件:(1) 无需知道真实参数,(2) 能够控制后验均值和方差,(3) 广泛适用于满足渐近贝叶斯中心极限定理条件的模型。事实上,即使在模型误设的情况下,我们的界依然有效。我们为包括逻辑回归在内的多种标准模型计算了界中的精确常数,并通过数值实验证明了其实用性。此外,我们还为更复杂模型的分析提供了理论框架。