Score function estimation is the cornerstone of both training and sampling from diffusion generative models. Despite this fact, the most commonly used estimators are either biased neural network approximations or high variance Monte Carlo estimators based on the conditional score. We introduce a novel nearest neighbour score function estimator which utilizes multiple samples from the training set to dramatically decrease estimator variance. We leverage our low variance estimator in two compelling applications. Training consistency models with our estimator, we report a significant increase in both convergence speed and sample quality. In diffusion models, we show that our estimator can replace a learned network for probability-flow ODE integration, opening promising new avenues of future research.

翻译：分数函数估计是扩散生成模型训练和采样的基石。然而，最常用的估计器要么是有偏的神经网络近似，要么是基于条件分数的高方差蒙特卡罗估计器。我们提出了一种新颖的最近邻分数函数估计器，利用训练集中的多个样本显著降低估计器方差。我们将这种低方差估计器应用于两个引人注目的场景：使用我们的估计器训练一致性模型，在收敛速度和样本质量上均实现了显著提升；在扩散模型中，我们证明该估计器可替代已学习的网络进行概率流常微分方程积分，为未来研究开辟了有前景的新方向。