In Bayesian inference, a widespread technique to approximately sample from and compute statistics of a high-dimensional posterior is to use the Laplace approximation, a Gaussian proxy to the posterior. The Laplace approximation accuracy improves as sample size grows, but the question of how fast dimension $d$ can grow with sample size $n$ has not been fully resolved. Prior works have shown that $d^3\ll n$ is a sufficient condition for accuracy of the approximation. But by deriving the leading order contribution to the TV error, we show that $d^2\ll n$ is sufficient. We show for a logistic regression posterior that this growth condition is necessary.
翻译:在贝叶斯推断中,一种广泛使用的技术是通过拉普拉斯近似(即用高斯分布近似后验)来近似采样并计算高维后验的统计量。拉普拉斯近似的精度随样本量增大而提升,但维度 $d$ 相对于样本量 $n$ 的增长速度问题尚未完全解决。已有研究证明,$d^3\ll n$ 是该近似精度的充分条件。然而,通过推导总变差误差的首阶贡献,我们证明了 $d^2\ll n$ 是充分的。针对逻辑回归后验,我们进一步表明这一增长条件是必要的。