In this study, we investigate stochastic optimization on Riemannian manifolds, focusing on the crucial variance reduction mechanism used in both Euclidean and Riemannian settings. Riemannian variance-reduced methods usually involve a double-loop structure, computing a full gradient at the start of each loop. Determining the optimal inner loop length is challenging in practice, as it depends on strong convexity or smoothness constants, which are often unknown or hard to estimate. Motivated by Euclidean methods, we introduce the Riemannian Loopless SVRG (R-LSVRG) and PAGE (R-PAGE) methods. These methods replace the outer loop with probabilistic gradient computation triggered by a coin flip in each iteration, ensuring simpler proofs, efficient hyperparameter selection, and sharp convergence guarantees. Using R-PAGE as a framework for non-convex Riemannian optimization, we demonstrate its applicability to various important settings. For example, we derive Riemannian MARINA (R-MARINA) for distributed settings with communication compression, providing the best theoretical communication complexity guarantees for non-convex distributed optimization over Riemannian manifolds. Experimental results support our theoretical findings.

翻译：本文研究黎曼流形上的随机优化问题，重点关注欧几里得与黎曼框架下均适用的关键方差缩减机制。黎曼方差缩减方法通常采用双循环结构，需在每个循环初始阶段计算全梯度。然而在实践中，最优内循环长度取决于强凸性或光滑性常数——这些参数往往未知或难以估计，导致其确定颇具挑战。受欧几里得方法的启发，我们提出黎曼无环SVRG（R-LSVRG）与黎曼无环PAGE（R-PAGE）方法。这两种方法通过每次迭代中基于抛硬币概率触发梯度计算来替代外循环，从而简化理论证明、实现高效超参数选择并保证严格的收敛速度。以R-PAGE作为非凸黎曼优化的通用框架，我们展示了其在多种重要场景中的适用性。例如，针对含通信压缩的分布式环境，我们推导出黎曼MARINA（R-MARINA）方法，为黎曼流形上的非凸分布式优化提供了理论通信复杂度最优保证。实验结果验证了理论分析。