Many machine learning problems can be formulated as approximating a target distribution using a particle distribution by minimizing a statistical discrepancy. Wasserstein Gradient Flow can be employed to move particles along a path that minimizes the $f$-divergence between the \textit{target} and \textit{particle} distributions. To perform such movements we need to calculate the corresponding velocity fields which include a density ratio function between these two distributions. While previous works estimated the density ratio function first and then differentiated the estimated ratio, this approach may suffer from overfitting, which leads to a less accurate estimate. Inspired by non-parametric curve fitting, we directly estimate these velocity fields using interpolation. We prove that our method is asymptotically consistent under mild conditions. We validate the effectiveness using novel applications on domain adaptation and missing data imputation.
翻译:许多机器学习问题可归结为通过最小化统计差异,利用粒子分布逼近目标分布。Wasserstein梯度流能够沿最小化目标分布与粒子分布之间$f$散度的路径移动粒子。为实现此类移动,需要计算相应速度场,该速度场包含两个分布间的密度比函数。现有方法虽先估计密度比函数再对其求导,但容易因过拟合导致估计精度欠佳。受非参数曲线拟合启发,本文直接通过插值法估计速度场。我们证明该方法在温和条件下具有渐近一致性,并通过领域自适应和缺失数据填补等创新应用验证其有效性。