Many Bayesian statistical inference problems come down to computing a maximum a-posteriori (MAP) assignment of latent variables. Yet, standard methods for estimating the MAP assignment do not have a finite time guarantee that the algorithm has converged to a fixed point. Previous research has found that MAP inference can be represented in dual form as a linear programming problem with a non-polynomial number of constraints. A Lagrangian relaxation of the dual yields a statistical inference algorithm as a linear programming problem. However, the decision as to which constraints to remove in the relaxation is often heuristic. We present a method for maximum a-posteriori inference in general Bayesian factor models that sequentially adds constraints to the fully relaxed dual problem using Benders' decomposition. Our method enables the incorporation of expressive integer and logical constraints in clustering problems such as must-link, cannot-link, and a minimum number of whole samples allocated to each cluster. Using this approach, we derive MAP estimation algorithms for the Bayesian Gaussian mixture model and latent Dirichlet allocation. Empirical results show that our method produces a higher optimal posterior value compared to Gibbs sampling and variational Bayes methods for standard data sets and provides certificate of convergence.

翻译：许多贝叶斯统计推断问题可归结为计算隐变量的最大后验（MAP）赋值。然而，估计MAP赋值的标准方法缺乏算法收敛到不动点的有限时间保证。先前研究发现，MAP推断可以表示为具有非多项式数量约束的线性规划问题的对偶形式。通过对偶问题的拉格朗日松弛，可将统计推断算法转化为线性规划问题。但松弛过程中约束去除的决策通常依赖启发式方法。本文提出一种适用于一般贝叶斯因子模型的最大后验推断方法，该方法利用Benders分解在完全松弛的对偶问题中逐步添加约束。我们的方法能够在聚类问题中融入丰富的整数约束与逻辑约束，例如必须链接约束、禁止链接约束以及每个簇最少完整样本数约束。基于该框架，我们推导出贝叶斯高斯混合模型和潜在狄利克雷分配的MAP估计算法。实验结果表明，在标准数据集上，相比吉布斯采样和变分贝叶斯方法，本方法能获得更优的后验值，并提供收敛性证明。