Recently, optimization on the Riemannian manifold has provided new insights to the optimization community. In this regard, the manifold taken as the probability measure metric space equipped with the second-order Wasserstein distance is of particular interest, since optimization on it can be linked to practical sampling processes. In general, the oracle (continuous) optimization method on Wasserstein space is Riemannian gradient flow (i.e., Langevin dynamics when minimizing KL divergence). In this paper, we aim to enrich the continuous optimization methods in the Wasserstein space by extending the gradient flow into the stochastic gradient descent (SGD) flow and stochastic variance reduction gradient (SVRG) flow. The two flows on Euclidean space are standard stochastic optimization methods, while their Riemannian counterparts are not explored yet. By leveraging the structures in Wasserstein space, we construct a stochastic differential equation (SDE) to approximate the discrete dynamics of desired stochastic methods in the corresponded random vector space. Then, the flows of probability measures are naturally obtained by applying Fokker-Planck equation to such SDE. Furthermore, the convergence rates of the proposed Riemannian stochastic flows are proven, and they match the results in Euclidean space.

翻译：近年来，黎曼流形上的优化为优化领域提供了新见解。其中，以二阶沃瑟斯坦距离为度量的概率测度度量空间作为流形具有特殊意义，因为该空间上的优化可关联至实际采样过程。通常，沃瑟斯坦空间上的理想（连续）优化方法是黎曼梯度流（即最小化KL散度时的朗之万动力学）。本文旨在通过将梯度流扩展为随机梯度下降（SGD）流和随机方差缩减梯度（SVRG）流，丰富沃瑟斯坦空间上的连续优化方法。这两种流在欧氏空间中是标准随机优化方法，但其黎曼对应形式尚未被探索。通过利用沃瑟斯坦空间的结构，我们构造了一个随机微分方程（SDE）以近似所需随机方法在对应随机向量空间中的离散动力学。随后，通过将福克-普朗克方程应用于该SDE，自然得到了概率测度流。此外，我们证明了所提黎曼随机流的收敛速率，且该结果与欧氏空间中的结果一致。