Recently, optimization on the Riemannian manifold has provided new insights to the optimization community. In this regard, the manifold taken as the probability measure metric space equipped with the second-order Wasserstein distance is of particular interest, since optimization on it can be linked to practical sampling processes. In general, the oracle (continuous) optimization method on Wasserstein space is Riemannian gradient flow (i.e., Langevin dynamics when minimizing KL divergence). In this paper, we aim to enrich the continuous optimization methods in the Wasserstein space by extending the gradient flow into the stochastic gradient descent (SGD) flow and stochastic variance reduction gradient (SVRG) flow. The two flows on Euclidean space are standard stochastic optimization methods, while their Riemannian counterparts are not explored yet. By leveraging the structures in Wasserstein space, we construct a stochastic differential equation (SDE) to approximate the discrete dynamics of desired stochastic methods in the corresponded random vector space. Then, the flows of probability measures are naturally obtained by applying Fokker-Planck equation to such SDE. Furthermore, the convergence rates of the proposed Riemannian stochastic flows are proven, and they match the results in Euclidean space.

翻译：近年来，黎曼流形上的优化问题为优化领域提供了新的视角。其中，配备二阶Wasserstein距离的概率测度度量空间作为流形尤为引人关注，因为在该空间上的优化可与实际采样过程建立联系。通常，Wasserstein空间上的预言机（连续）优化方法为黎曼梯度流（即最小化KL散度时的Langevin动力学）。本文旨在将梯度流扩展至随机梯度下降（SGD）流与随机方差缩减梯度（SVRG）流，从而丰富Wasserstein空间中的连续优化方法。欧氏空间中的这两种流是标准的随机优化方法，但其黎曼对应形式尚未被探索。通过利用Wasserstein空间的结构，我们构建了一个随机微分方程（SDE）来逼近所需随机方法在对应随机向量空间中的离散动力学。进而，通过将Fokker-Planck方程应用于该SDE，自然得到了概率测度的流。此外，本文证明了所提出的黎曼随机流的收敛速率，其结果与欧氏空间中的结论相匹配。