Recently, zhang et al have proposed the Diffusion Exponential Integrator Sampler (DEIS) for fast generation of samples from Diffusion Models. It leverages the semi-linear nature of the probability flow ordinary differential equation (ODE) in order to greatly reduce integration error and improve generation quality at low numbers of function evaluations (NFEs). Key to this approach is the score function reparameterisation, which reduces the integration error incurred from using a fixed score function estimate over each integration step. The original authors use the default parameterisation used by models trained for noise prediction -- multiply the score by the standard deviation of the conditional forward noising distribution. We find that although the mean absolute value of this score parameterisation is close to constant for a large portion of the reverse sampling process, it changes rapidly at the end of sampling. As a simple fix, we propose to instead reparameterise the score (at inference) by dividing it by the average absolute value of previous score estimates at that time step collected from offline high NFE generations. We find that our score normalisation (DEIS-SN) consistently improves FID compared to vanilla DEIS, showing an FID improvement from 6.44 to 5.57 at 10 NFEs for our CIFAR-10 experiments. Our code is available at https://github.com/mtkresearch/Diffusion-DEIS-SN.

翻译：最近，Zhang等人提出了扩散指数积分器采样器（DEIS），用于从扩散模型中快速生成样本。该方法利用概率流常微分方程（ODE）的半线性特性，在少量函数评估（NFE）下显著减少积分误差并提升生成质量。其核心在于得分函数的重参数化，该策略通过降低每个积分步骤中使用固定得分估计所产生的积分误差。原论文作者采用了为噪声预测训练的模型默认参数化方案——将得分乘以条件前向噪声分布的标准差。我们发现，尽管该得分参数化的平均绝对值在反向采样过程的大部分区间内接近常数，但在采样末期会快速变化。作为简单改进，我们提出在推理阶段通过将得分除以离线高NFE生成中同一时间步先前得分估计的平均绝对值来重新参数化得分。实验表明，我们的得分归一化方法（DEIS-SN）相较于原始DEIS持续改进了FID值：在CIFAR-10数据集上，10次NFE时FID从6.44提升至5.57。代码已开源至https://github.com/mtkresearch/Diffusion-DEIS-SN。