Diffusion bridges are a promising class of deep-learning methods for sampling from unnormalized distributions. Recent works show that the Log Variance (LV) loss consistently outperforms the reverse Kullback-Leibler (rKL) loss when using the reparametrization trick to compute rKL-gradients. While the on-policy LV loss yields identical gradients to the rKL loss when combined with the log-derivative trick for diffusion samplers with non-learnable forward processes, this equivalence does not hold for diffusion bridges or when diffusion coefficients are learned. Based on this insight we argue that for diffusion bridges the LV loss does not represent an optimization objective that can be motivated like the rKL loss via the data processing inequality. Our analysis shows that employing the rKL loss with the log-derivative trick (rKL-LD) does not only avoid these conceptual problems but also consistently outperforms the LV loss. Experimental results with different types of diffusion bridges on challenging benchmarks show that samplers trained with the rKL-LD loss achieve better performance. From a practical perspective we find that rKL-LD requires significantly less hyperparameter optimization and yields more stable training behavior.

翻译：扩散桥是一类有前景的深度学习方法，用于从非归一化分布中采样。近期研究表明，当使用重参数化技巧计算rKL梯度时，对数方差损失在性能上始终优于反向Kullback-Leibler损失。虽然对于具有不可学习前向过程的扩散采样器，基于策略的对数方差损失与结合对数导数技巧的rKL损失会产生相同的梯度，但这种等价性在扩散桥或扩散系数可学习的情况下并不成立。基于这一发现，我们认为对于扩散桥而言，对数方差损失并不能像通过数据处理不等式推导的rKL损失那样构成具有理论动机的优化目标。我们的分析表明，采用结合对数导数技巧的rKL损失不仅能够避免这些概念性问题，而且在性能上持续优于对数方差损失。在具有挑战性的基准测试中，使用不同类型扩散桥的实验结果表明，采用rKL-LD损失训练的采样器实现了更优的性能。从实践角度来看，我们发现rKL-LD需要显著更少的超参数优化，并能产生更稳定的训练行为。