Riemannian submanifold optimization with momentum is computationally challenging because ensuring iterates remain on the submanifold often requires solving difficult differential equations. We simplify such optimization algorithms for the submanifold of symmetric positive-definite matrices with the affine invariant metric. We propose a generalized version of the Riemannian normal coordinates which dynamically trivializes the problem into a Euclidean unconstrained problem. We use our approach to explain and simplify existing approaches for structured covariances and develop efficient second-order optimizers for deep learning without explicit matrix inverses.

翻译：带动量的黎曼子流形优化在计算上具有挑战性，因为确保迭代点始终保持在子流形上通常需要求解复杂的微分方程。针对具有仿射不变度量的对称正定矩阵子流形，我们简化了此类优化算法。提出了一种广义的黎曼法坐标，该坐标将问题动态平凡化为一个无约束的欧几里得问题。利用该方法，我们解释并简化了现有针对结构化协方差的方法，并开发了无需显式矩阵求逆的高效深度学习二阶优化器。