This work puts forth low-complexity Riemannian subspace descent algorithms for the minimization of functions over the symmetric positive definite (SPD) manifold. Different from the existing Riemannian gradient descent variants, the proposed approach utilizes carefully chosen subspaces that allow the update to be written as a product of the Cholesky factor of the iterate and a sparse matrix. The resulting updates avoid the costly matrix operations like matrix exponentiation and dense matrix multiplication, which are generally required in almost all other Riemannian optimization algorithms on SPD manifold. We further identify a broad class of functions, arising in diverse applications, such as kernel matrix learning, covariance estimation of Gaussian distributions, maximum likelihood parameter estimation of elliptically contoured distributions, and parameter estimation in Gaussian mixture model problems, over which the Riemannian gradients can be calculated efficiently. The proposed uni-directional and multi-directional Riemannian subspace descent variants incur per-iteration complexities of $O(n)$ and $O(n^2)$ respectively, as compared to the $O(n^3)$ or higher complexity incurred by all existing Riemannian gradient descent variants. The superior runtime and low per-iteration complexity of the proposed algorithms is also demonstrated via numerical tests on large-scale covariance estimation and matrix square root problems. MATLAB code implementation is publicly available on GitHub : https://github.com/yogeshd-iitk/subspace_descent_over_SPD_manifold

翻译：本文提出了一种低复杂度的黎曼子空间下降算法，用于在对称正定（SPD）流形上最小化函数。与现有的黎曼梯度下降变体不同，所提出的方法利用精心选择的子空间，使得迭代更新可以表示为迭代的Cholesky因子与稀疏矩阵的乘积。这种更新避免了几乎所有其他SPD流形上的黎曼优化算法通常需要的昂贵矩阵运算，如矩阵指数化和稠密矩阵乘法。我们进一步识别出一类广泛的函数，这些函数出现在各种应用中，例如核矩阵学习、高斯分布的协方差估计、椭圆分布的最大似然参数估计以及高斯混合模型问题中的参数估计，在这些函数上，黎曼梯度可以高效计算。所提出的单向和多向黎曼子空间下降变体每次迭代的复杂度分别为$O(n)$和$O(n^2)$，而所有现有的黎曼梯度下降变体都需要$O(n^3)$或更高的复杂度。通过在大规模协方差估计和矩阵平方根问题上的数值测试，也展示了所提算法的优越运行时间和低每步复杂度。MATLAB代码实现已在GitHub上公开：https://github.com/yogeshd-iitk/subspace_descent_over_SPD_manifold