The expectation maximization (EM) algorithm is a widespread method for empirical Bayesian inference, but its expectation step (E-step) is often intractable. Employing a stochastic approximation scheme with Markov chain Monte Carlo (MCMC) can circumvent this issue, resulting in an algorithm known as MCMC-SAEM. While theoretical guarantees for MCMC-SAEM have previously been established, these results are restricted to the case where asymptotically unbiased MCMC algorithms are used. In practice, MCMC-SAEM is often run with asymptotically biased MCMC, for which the consequences are theoretically less understood. In this work, we fill this gap by analyzing the asymptotics and non-asymptotics of SAEM with biased MCMC steps, particularly the effect of bias. We also provide numerical experiments comparing the Metropolis-adjusted Langevin algorithm (MALA), which is asymptotically unbiased, and the unadjusted Langevin algorithm (ULA), which is asymptotically biased, on synthetic and real datasets. Experimental results show that ULA is more stable with respect to the choice of Langevin stepsize and can sometimes result in faster convergence.

翻译：期望最大化（EM）算法是经验贝叶斯推断中广泛使用的方法，但其期望步骤（E步）往往难以求解。采用马尔可夫链蒙特卡洛（MCMC）的随机近似方案可规避这一问题，由此衍生出MCMC-SAEM算法。尽管已有研究为MCMC-SAEM提供了理论保障，但这些结论仅适用于使用渐近无偏MCMC算法的情况。在实际应用中，MCMC-SAEM常与渐近有偏MCMC配合使用，此类情况的理论后果尚缺乏深入理解。本文通过分析采用有偏MCMC步骤的SAEM算法的渐近性与非渐近性，特别是偏差效应，填补了这一理论空白。我们还对合成数据集与真实数据集进行了数值实验，比较了渐近无偏的Metropolis调整Langevin算法（MALA）与渐近有偏的未调整Langevin算法（ULA）。实验结果表明，ULA对Langevin步长的选择更具鲁棒性，且在某些情况下能实现更快的收敛速度。