Recently, Zhang et al. have proposed the Diffusion Exponential Integrator Sampler (DEIS) for fast generation of samples from Diffusion Models. It leverages the semi-linear nature of the probability flow ordinary differential equation (ODE) in order to greatly reduce integration error and improve generation quality at low numbers of function evaluations (NFEs). Key to this approach is the score function reparameterisation, which reduces the integration error incurred from using a fixed score function estimate over each integration step. The original authors use the default parameterisation used by models trained for noise prediction -- multiply the score by the standard deviation of the conditional forward noising distribution. We find that although the mean absolute value of this score parameterisation is close to constant for a large portion of the reverse sampling process, it changes rapidly at the end of sampling. As a simple fix, we propose to instead reparameterise the score (at inference) by dividing it by the average absolute value of previous score estimates at that time step collected from offline high NFE generations. We find that our score normalisation (DEIS-SN) consistently improves FID compared to vanilla DEIS, showing an improvement at 10 NFEs from 6.44 to 5.57 on CIFAR-10 and from 5.9 to 4.95 on LSUN-Church 64x64. Our code is available at https://github.com/mtkresearch/Diffusion-DEIS-SN

翻译：近日，Zhang等人提出了扩散指数积分采样器（DEIS），用于快速生成扩散模型样本。该方法利用概率流常微分方程（ODE）的半线性特性，显著降低积分误差，并在低函数评估次数（NFEs）下提升生成质量。该技术的核心在于得分函数重参数化，通过减少每个积分步骤中固定得分函数估计引起的积分误差。原作者采用噪声预测训练模型的默认参数化方式——即用条件前向噪声分布的标准差乘以得分。我们发现，尽管该得分参数化的平均绝对值在反向采样过程的大部分区间接近常数，但在采样结束时会发生急剧变化。作为简单改进，我们提出在推理阶段对得分进行重参数化，将其除以从离线高NFE生成中收集的该时间步历史得分估计的平均绝对值。实验表明，相较于标准DEIS，我们的得分归一化方法（DEIS-SN）持续改善了FID指标：在CIFAR-10数据集10次NFE下，FID从6.44降至5.57；在LSUN-Church 64x64数据集上，FID从5.9降至4.95。我们的代码已开源：https://github.com/mtkresearch/Diffusion-DEIS-SN