Block majorization-minimization (BMM) is a simple iterative algorithm for nonconvex constrained optimization that sequentially minimizes majorizing surrogates of the objective function in each block coordinate while the other coordinates are held fixed. BMM entails a large class of optimization algorithms such as block coordinate descent and its proximal-point variant, expectation-minimization, and block projected gradient descent. We establish that for general constrained nonconvex optimization, BMM with strongly convex surrogates can produce an $\epsilon$-stationary point within $O(\epsilon^{-2}(\log \epsilon^{-1})^{2})$ iterations and asymptotically converges to the set of stationary points. Furthermore, we propose a trust-region variant of BMM that can handle surrogates that are only convex and still obtain the same iteration complexity and asymptotic stationarity. These results hold robustly even when the convex sub-problems are inexactly solved as long as the optimality gaps are summable. As an application, we show that a regularized version of the celebrated multiplicative update algorithm for nonnegative matrix factorization by Lee and Seung has iteration complexity of $O(\epsilon^{-2}(\log \epsilon^{-1})^{2})$. The same result holds for a wide class of regularized nonnegative tensor decomposition algorithms as well as the classical block projected gradient descent algorithm. These theoretical results are validated through various numerical experiments.

翻译：块主化-最小化算法是一种针对非凸约束优化的简单迭代算法，其通过在其他坐标固定时，依次最小化目标函数在每个块坐标上的主化替代函数。该算法涵盖了包括块坐标下降法及其近邻点变体、期望最大化算法以及块投影梯度下降法在内的大类优化算法。我们证明，对于一般约束非凸优化问题，采用强凸替代函数的块主化-最小化算法可在$O(\epsilon^{-2}(\log \epsilon^{-1})^{2})$次迭代内生成$\epsilon$-稳定点，且渐近收敛于稳定点集。此外，我们提出一种信赖域变体，该变体能处理仅具凸性的替代函数，同时仍保持相同的迭代复杂度与渐近稳定性。即使凸子问题因最优性间隙可加和而未被精确求解，上述结论仍具有鲁棒性。作为应用实例，我们证明李与承提出的非负矩阵分解经典乘法更新算法的正则化版本具有$O(\epsilon^{-2}(\log \epsilon^{-1})^{2})$的迭代复杂度。该结论同样适用于宽泛的正则化非负张量分解算法以及经典块投影梯度下降算法。这些理论结果通过多种数值实验得到验证。