Latent Gaussian models have a rich history in statistics and machine learning, with applications ranging from factor analysis to compressed sensing to time series analysis. The classical method for maximizing the likelihood of these models is the expectation-maximization (EM) algorithm. For problems with high-dimensional latent variables and large datasets, EM scales poorly because it needs to invert as many large covariance matrices as the number of data points. We introduce probabilistic unrolling, a method that combines Monte Carlo sampling with iterative linear solvers to circumvent matrix inversion. Our theoretical analyses reveal that unrolling and backpropagation through the iterations of the solver can accelerate gradient estimation for maximum likelihood estimation. In experiments on simulated and real data, we demonstrate that probabilistic unrolling learns latent Gaussian models up to an order of magnitude faster than gradient EM, with minimal losses in model performance.

翻译：潜在高斯模型在统计学与机器学习领域具有悠久历史，其应用涵盖因子分析、压缩感知及时间序列分析等多个方向。经典的最大化似然估计方法为期望最大化(EM)算法。然而，对于高维潜在变量及大规模数据集问题，EM算法因需要为每个数据点求解大型协方差矩阵的逆矩阵而扩展性极差。本文提出概率展开方法，通过结合蒙特卡洛采样与迭代线性求解器来规避矩阵求逆。理论分析表明，在求解器的迭代过程中进行展开与反向传播可加速最大似然估计的梯度计算。在模拟与真实数据上的实验证明，与梯度EM算法相比，概率展开方法能以更快的速度（最高提升一个数量级）学习潜在高斯模型，且模型性能损失极小。