Block majorization-minimization (BMM) is a simple iterative algorithm for nonconvex constrained optimization that sequentially minimizes majorizing surrogates of the objective function in each block coordinate while the other coordinates are held fixed. BMM entails a large class of optimization algorithms such as block coordinate descent and its proximal-point variant, expectation-minimization, and block projected gradient descent. We establish that for general constrained nonconvex optimization, BMM with strongly convex surrogates can produce an $\epsilon$-stationary point within $O(\epsilon^{-2}(\log \epsilon^{-1})^{2})$ iterations and asymptotically converges to the set of stationary points. Furthermore, we propose a trust-region variant of BMM that can handle surrogates that are only convex and still obtain the same iteration complexity and asymptotic stationarity. These results hold robustly even when the convex sub-problems are inexactly solved as long as the optimality gaps are summable. As an application, we show that a regularized version of the celebrated multiplicative update algorithm for nonnegative matrix factorization by Lee and Seung has iteration complexity of $O(\epsilon^{-2}(\log \epsilon^{-1})^{2})$. The same result holds for a wide class of regularized nonnegative tensor decomposition algorithms as well as the classical block projected gradient descent algorithm. These theoretical results are validated through various numerical experiments.

翻译：块状主极小化（BMM）是一种用于非凸约束优化的简单迭代算法，它在固定其他坐标的同时，依次最小化每个块坐标中目标函数的主化代理。BMM涵盖了一大类优化算法，如块坐标下降法及其近端点变体、期望最大化算法和块投影梯度下降法。我们证明，对于一般的约束非凸优化，采用强凸代理的BMM可以在$O(\epsilon^{-2}(\log \epsilon^{-1})^{2})$次迭代内产生一个$\epsilon$-稳定点，并渐近收敛到稳定点集合。此外，我们提出了一种BMM的信赖域变体，该变体能够处理仅凸的代理，同时仍能获得相同的迭代复杂度和渐近稳定性。当凸子问题非精确求解时，只要最优性间隙是可和的，这些结果仍然稳健成立。作为应用，我们证明了Lee和Seung提出的用于非负矩阵分解的著名乘法更新算法的正则化版本具有$O(\epsilon^{-2}(\log \epsilon^{-1})^{2})$的迭代复杂度。同样的结果也适用于一大类正则化非负张量分解算法以及经典的块投影梯度下降算法。这些理论结果通过多种数值实验得到了验证。