We analyze inexact Riemannian gradient descent (RGD) where Riemannian gradients and retractions are inexactly (and cheaply) computed. Our focus is on understanding when inexact RGD converges and what is the complexity in the general nonconvex and constrained setting. We answer these questions in a general framework of tangential Block Majorization-Minimization (tBMM). We establish that tBMM converges to an $\epsilon$-stationary point within $O(\epsilon^{-2})$ iterations. Under a mild assumption, the results still hold when the subproblem is solved inexactly in each iteration provided the total optimality gap is bounded. Our general analysis applies to a wide range of classical algorithms with Riemannian constraints including inexact RGD and proximal gradient method on Stiefel manifolds. We numerically validate that tBMM shows improved performance over existing methods when applied to various problems, including nonnegative tensor decomposition with Riemannian constraints, regularized nonnegative matrix factorization, and low-rank matrix recovery problems.

翻译：我们分析了非精确黎曼梯度下降（RGD）方法，其中黎曼梯度和回缩操作被非精确（且低成本）地计算。本文重点研究非精确RGD在何种条件下收敛，以及其在一般非凸约束环境下的复杂度。我们在切向块极大化-极小化（tBMM）的通用框架下回答了这些问题。我们证明tBMM可在$O(\epsilon^{-2})$次迭代内收敛至$\epsilon$-稳定点。在温和假设下，若每次迭代中子问题仅被非精确求解且总最优性间隙有界，该结论仍成立。我们的通用分析适用于多种含黎曼约束的经典算法，包括斯提弗尔流形上的非精确RGD和邻近梯度法。通过数值实验验证，tBMM在多种问题（包括非负张量分解、正则化非负矩阵分解以及低秩矩阵恢复）中均展现出优于现有方法的性能。