Sampling a probability distribution with an unknown normalization constant is a fundamental problem in computational science and engineering. This task may be cast as an optimization problem over all probability measures, and an initial distribution can be evolved to the desired minimizer dynamically via gradient flows. Mean-field models, whose law is governed by the gradient flow in the space of probability measures, may also be identified; particle approximations of these mean-field models form the basis of algorithms. The gradient flow approach is also the basis of algorithms for variational inference, in which the optimization is performed over a parameterized family of probability distributions such as Gaussians, and the underlying gradient flow is restricted to the parameterized family. By choosing different energy functionals and metrics for the gradient flow, different algorithms with different convergence properties arise. In this paper, we concentrate on the Kullback-Leibler divergence after showing that, up to scaling, it has the unique property that the gradient flows resulting from this choice of energy do not depend on the normalization constant. For the metrics, we focus on variants of the Fisher-Rao, Wasserstein, and Stein metrics; we introduce the affine invariance property for gradient flows, and their corresponding mean-field models, determine whether a given metric leads to affine invariance, and modify it to make it affine invariant if it does not. We study the resulting gradient flows in both probability density space and Gaussian space. The flow in the Gaussian space may be understood as a Gaussian approximation of the flow. We demonstrate that the Gaussian approximation based on the metric and through moment closure coincide, establish connections between them, and study their long-time convergence properties showing the advantages of affine invariance.

翻译：对未知归一化常数的概率分布进行采样是计算科学与工程中的基本问题。该任务可被转化为所有概率测度上的优化问题，通过梯度流可动态地将初始分布演化为目标极小化分布。可识别出以概率测度空间中的梯度流为控制律的平均场模型，这些平均场模型的粒子近似构成了算法的基础。梯度流方法也是变分推断算法的基础——在变分推断中，优化在参数化概率分布族（如高斯分布）上进行，梯度流被约束在参数化族内。通过选择不同的能量泛函和梯度流度量，可得到具有不同收敛性质的不同算法。本文在证明KL散度（在缩放意义下）具有唯一性质——即由该能量选择产生的梯度流不依赖于归一化常数——之后，聚焦于KL散度的研究。对于度量选择，我们重点关注Fisher-Rao度量、Wasserstein度量和Stein度量的变体；引入梯度流及其对应平均场模型的仿射不变性，判定给定度量是否具有仿射不变性，并对不具备该性质的度量进行改造使其满足仿射不变性。我们在概率密度空间和高斯空间中分别研究由此产生的梯度流。高斯空间中的流可被理解为流的近似。我们证明了基于度量的高斯近似与通过矩闭合获得的高斯近似具有一致性，建立了二者的关联，并通过长时收敛性质的研究展示了仿射不变性的优势。