This paper introduces Bespoke Non-Stationary (BNS) Solvers, a solver distillation approach to improve sample efficiency of Diffusion and Flow models. BNS solvers are based on a family of non-stationary solvers that provably subsumes existing numerical ODE solvers and consequently demonstrate considerable improvement in sample approximation (PSNR) over these baselines. Compared to model distillation, BNS solvers benefit from a tiny parameter space ($<$200 parameters), fast optimization (two orders of magnitude faster), maintain diversity of samples, and in contrast to previous solver distillation approaches nearly close the gap from standard distillation methods such as Progressive Distillation in the low-medium NFE regime. For example, BNS solver achieves 45 PSNR / 1.76 FID using 16 NFE in class-conditional ImageNet-64. We experimented with BNS solvers for conditional image generation, text-to-image generation, and text-2-audio generation showing significant improvement in sample approximation (PSNR) in all.

翻译：本文提出定制非稳态（BNS）求解器，这是一种通过求解器蒸馏来提升扩散模型与流模型采样效率的新方法。BNS求解器基于一类非稳态求解器族，该类求解器在理论上覆盖了现有数值常微分方程求解器的适用范围，因此在样本逼近性能（PSNR指标）上显著优于这些基线方法。与模型蒸馏相比，BNS求解器具有参数空间极小（<200个参数）、优化速度快（提升两个数量级）且能保持样本多样性的优势。此外，与以往的求解器蒸馏方法不同，BNS求解器在中低NFE（函数评估次数）区间内几乎弥合了与渐进蒸馏等标准蒸馏方法的性能差距。例如，在类别条件ImageNet-64数据集上，BNS求解器通过16次NFE实现了45 PSNR / 1.76 FID的性能。我们已将BNS求解器应用于条件图像生成、文本到图像生成及文本到音频生成任务，验证了其在所有任务中对样本逼近性能（PSNR）的显著提升。