We present Zeroth-order Riemannian Averaging Stochastic Approximation (\texttt{Zo-RASA}) algorithms for stochastic optimization on Riemannian manifolds. We show that \texttt{Zo-RASA} achieves optimal sample complexities for generating $\epsilon$-approximation first-order stationary solutions using only one-sample or constant-order batches in each iteration. Our approach employs Riemannian moving-average stochastic gradient estimators, and a novel Riemannian-Lyapunov analysis technique for convergence analysis. We improve the algorithm's practicality by using retractions and vector transport, instead of exponential mappings and parallel transports, thereby reducing per-iteration complexity. Additionally, we introduce a novel geometric condition, satisfied by manifolds with bounded second fundamental form, which enables new error bounds for approximating parallel transport with vector transport.

翻译：我们提出了零阶黎曼平均随机逼近（\texttt{Zo-RASA}）算法，用于黎曼流形上的随机优化。我们证明，通过在每次迭代中仅使用单样本或常数阶的批次，\texttt{Zo-RASA}能够达到生成$\epsilon$-近似一阶驻点所需的最优样本复杂度。该方法采用了黎曼移动平均随机梯度估计器，并引入了一种新的黎曼-李雅普诺夫分析技术用于收敛性分析。我们通过使用收缩与向量传输代替指数映射与平行传输，降低了每次迭代的复杂度，从而提升了算法的实用性。此外，我们引入了一个新的几何条件（适用于具有有界第二基本形式的流形），该条件能够为利用向量传输逼近平行传输提供新的误差界。