Many machine learning applications are naturally formulated as optimization problems on Riemannian manifolds. The main idea behind Riemannian optimization is to maintain the feasibility of the variables while moving along a descent direction on the manifold. This results in updating all the variables at every iteration. In this work, we provide a general framework for developing computationally efficient coordinate descent (CD) algorithms on matrix manifolds that allows updating only a few variables at every iteration while adhering to the manifold constraint. In particular, we propose CD algorithms for various manifolds such as Stiefel, Grassmann, (generalized) hyperbolic, symplectic, and symmetric positive (semi)definite. While the cost per iteration of the proposed CD algorithms is low, we further develop a more efficient variant via a first-order approximation of the objective function. We analyze their convergence and complexity, and empirically illustrate their efficacy in several applications.

翻译：许多机器学习应用自然地表述为黎曼流形上的优化问题。黎曼优化的核心思想是在保持变量可行性的同时，沿流形上的下降方向移动。这导致每次迭代需要更新所有变量。本文提出了一个通用框架，用于在矩阵流形上开发计算高效的坐标下降算法，该框架允许每次迭代仅更新少量变量，同时满足流形约束。具体而言，我们针对多种流形提出了坐标下降算法，例如Stiefel流形、Grassmann流形、（广义）双曲流形、辛流形以及对称正定（半正定）流形。尽管所提坐标下降算法的单次迭代成本较低，我们通过目标函数的一阶近似进一步开发了更高效的变体。我们分析了这些算法的收敛性和计算复杂度，并通过多个应用实例实证展示了其有效性。