This paper investigates gradient descent for solving low-rank matrix approximation problems. We begin by establishing the local linear convergence of gradient descent for symmetric matrix approximation. Building on this result, we prove the rapid global convergence of gradient descent, particularly when initialized with small random values. Remarkably, we show that even with moderate random initialization, which includes small random initialization as a special case, gradient descent achieves fast global convergence in scenarios where the top eigenvalues are identical. Furthermore, we extend our analysis to address asymmetric matrix approximation problems and investigate the effectiveness of a retraction-free eigenspace computation method. Numerical experiments strongly support our theory. In particular, the retraction-free algorithm outperforms the corresponding Riemannian gradient descent method, resulting in a significant 29\% reduction in runtime.

翻译：本文研究了用于求解低秩矩阵逼近问题的梯度下降方法。我们首先建立了对称矩阵逼近中梯度下降的局部线性收敛性。基于这一结果，我们证明了梯度下降的快速全局收敛性，特别是在采用小随机初始化的情况下。值得注意的是，我们表明即使在中等随机初始化（小随机初始化是其特例）的情况下，当顶部特征值相同时，梯度下降仍能实现快速全局收敛。此外，我们将分析扩展到非对称矩阵逼近问题，并研究了一种无回缩特征空间计算方法的有效性。数值实验有力支持了我们的理论。特别地，无回缩方法优于相应的黎曼梯度下降方法，实现了29%的运行时间显著减少。