While systems analysis has been studied for decades in the context of control theory, it has only been recently used to improve the convergence of Denoising Diffusion Probabilistic Models. This work describes a novel improvement to Third- Order Langevin Dynamics (TOLD), a recent diffusion method that performs better than its predecessors. This improvement, abbreviated TOLD++, is carried out by critically damping the TOLD forward transition matrix similarly to Dockhorn's Critically-Damped Langevin Dynamics (CLD). Specifically, it exploits eigen-analysis of the forward transition matrix to derive the optimal set of dynamics under the original TOLD scheme. TOLD++ is theoretically guaranteed to converge faster than TOLD, and its faster convergence is verified on the Swiss Roll toy dataset and CIFAR-10 dataset according to the FID metric.
翻译:尽管系统分析在控制理论背景下已研究数十年,但直到近期才被用于改进去噪扩散概率模型的收敛性。本研究提出对三阶朗之万动力学(TOLD)的创新改进,该扩散方法在性能上优于先前方法。此项改进(简称TOLD++)通过对TOLD前向转移矩阵实施临界阻尼实现,其原理类似于Dockhorn的临界阻尼朗之万动力学(CLD)。具体而言,该方法利用前向转移矩阵的特征分析,推导出原始TOLD框架下的最优动力学参数集。理论证明TOLD++具有比TOLD更快的收敛速度,在Swiss Roll玩具数据集和CIFAR-10数据集上,根据FID指标验证了其加速收敛的特性。