We consider gradient-related methods for low-rank matrix optimization with a smooth cost function. The methods operate on single factors of the low-rank factorization and share aspects of both alternating and Riemannian optimization. Two possible choices for the search directions based on Gauss-Southwell type selection rules are compared: one using the gradient of a factorized non-convex formulation, the other using the Riemannian gradient. While both methods provide gradient convergence guarantees that are similar to the unconstrained case, the version based on Riemannian gradient is significantly more robust with respect to small singular values and the condition number of the cost function, as illustrated by numerical experiments. As a side result of our approach, we also obtain new convergence results for the alternating least squares method.

翻译：考虑带光滑代价函数的低秩矩阵优化问题的梯度相关方法。这些方法作用于低秩分解的单个因子，兼具交替优化与黎曼优化的特点。比较了两种基于高斯-索斯韦尔型选择规则的搜索方向：一种采用因子化非凸公式的梯度，另一种采用黎曼梯度。尽管两种方法均能提供与无约束情形相似的梯度收敛保证，但数值实验表明，基于黎曼梯度的版本在应对小奇异值和代价函数条件数方面显著更稳健。作为本研究的附带成果，我们还获得了交替最小二乘法的新收敛结果。