Stein Variational Gradient Descent (SVGD) can transport particles along trajectories that reduce the KL divergence between the target and particle distribution but requires the target score function to compute the update. We introduce a new perspective on SVGD that views it as a local estimator of the reversed KL gradient flow. This perspective inspires us to propose new estimators that use local linear models to achieve the same purpose. The proposed estimators can be computed using only samples from the target and particle distribution without needing the target score function. Our proposed variational gradient estimators utilize local linear models, resulting in computational simplicity while maintaining effectiveness comparable to SVGD in terms of estimation biases. Additionally, we demonstrate that under a mild assumption, the estimation of high-dimensional gradient flow can be translated into a lower-dimensional estimation problem, leading to improved estimation accuracy. We validate our claims with experiments on both simulated and real-world datasets.

翻译：Stein变分梯度下降（SVGD）通过沿减小目标分布与粒子分布之间KL散度的轨迹传输粒子，但其更新过程需要目标得分函数。我们提出SVGD的新视角，将其视为反向KL梯度流的局部估计器。该视角启发我们提出利用局部线性模型实现相同目标的新估计器。所提出的估计器仅需目标分布与粒子分布的样本即可计算，无需目标得分函数。我们提出的变分梯度估计器采用局部线性模型，在保持与SVGD相当的估计偏差有效性的同时，实现了计算简洁性。此外，我们证明在温和假设下，高维梯度流的估计可转化为低维估计问题，从而提升估计精度。我们通过模拟和真实数据集的实验验证了上述论断。