This work puts forth low-complexity Riemannian subspace descent algorithms for the minimization of functions over the symmetric positive definite (SPD) manifold. Different from the existing Riemannian gradient descent variants, the proposed approach utilizes carefully chosen subspaces that allow the update to be written as a product of the Cholesky factor of the iterate and a sparse matrix. The resulting updates avoid the costly matrix operations like matrix exponentiation and dense matrix multiplication, which are generally required in almost all other Riemannian optimization algorithms on SPD manifold. We further identify a broad class of functions, arising in diverse applications, such as kernel matrix learning, covariance estimation of Gaussian distributions, maximum likelihood parameter estimation of elliptically contoured distributions, and parameter estimation in Gaussian mixture model problems, over which the Riemannian gradients can be calculated efficiently. The proposed uni-directional and multi-directional Riemannian subspace descent variants incur per-iteration complexities of $\O(n)$ and $\O(n^2)$ respectively, as compared to the $\O(n^3)$ or higher complexity incurred by all existing Riemannian gradient descent variants. The superior runtime and low per-iteration complexity of the proposed algorithms is also demonstrated via numerical tests on large-scale covariance estimation and matrix square root problems.

翻译：摘要：本文针对对称正定（SPD）流形上的函数最小化问题，提出了一种低复杂度的黎曼子空间下降算法。与现有黎曼梯度下降变体不同，所提方法利用精心选取的子空间，使得更新可表示为迭代点Cholesky因子与稀疏矩阵的乘积。该更新避免了几乎所有其他SPD流形黎曼优化算法通常所需的矩阵指数运算和稠密矩阵乘法等高成本矩阵操作。我们进一步在核矩阵学习、高斯分布协方差估计、椭圆等高分布极大似然参数估计以及高斯混合模型参数估计等多元应用场景中，识别出一类可高效计算黎曼梯度的广泛函数。所提出的单方向与多方向黎曼子空间下降变体每次迭代的复杂度分别为$\O(n)$和$\O(n^2)$，而现有黎曼梯度下降变体均需$\O(n^3)$或更高复杂度。通过大规模协方差估计与矩阵平方根问题的数值实验，验证了所提算法在运行时间和低单步复杂度方面的优越性能。