We consider gradient-related methods for low-rank matrix optimization with a smooth cost function. The methods operate on single factors of the low-rank factorization and share aspects of both alternating and Riemannian optimization. Two possible choices for the search directions based on Gauss-Southwell type selection rules are compared: one using the gradient of a factorized non-convex formulation, the other using the Riemannian gradient. While both methods provide gradient convergence guarantees that are similar to the unconstrained case, numerical experiments on a quadratic cost function indicate that the version based on the Riemannian gradient is significantly more robust with respect to small singular values and the condition number of the cost function. As a side result of our approach, we also obtain new convergence results for the alternating least squares method.

翻译：我们考虑针对具有光滑代价函数的低秩矩阵优化的梯度相关方法。这些方法作用于低秩分解的单个因子，融合了交替优化与黎曼优化的特征。基于Gauss-Southwell型选择准则，我们比较了两种可能的搜索方向：一种利用非凸分解形式的梯度，另一种利用黎曼梯度。尽管两种方法均能提供与无约束情形相似的梯度收敛保证，但针对二次代价函数的数值实验表明，基于黎曼梯度的版本在小奇异值和代价函数条件数方面具有显著更强的鲁棒性。作为本方法的附加成果，我们还获得了交替最小二乘法的新收敛结果。