The aim of this short note is to show that Denoising Diffusion Probabilistic Model DDPM, a non-homogeneous discrete-time Markov process, can be represented by a time-homogeneous continuous-time Markov process observed at non-uniformly sampled discrete times. Surprisingly, this continuous-time Markov process is the well-known and well-studied Ornstein-Ohlenbeck (OU) process, which was developed in 1930's for studying Brownian particles in Harmonic potentials. We establish the formal equivalence between DDPM and the OU process using its analytical solution. We further demonstrate that the design problem of the noise scheduler for non-homogeneous DDPM is equivalent to designing observation times for the OU process. We present several heuristic designs for observation times based on principled quantities such as auto-variance and Fisher Information and connect them to ad hoc noise schedules for DDPM. Interestingly, we show that the Fisher-Information-motivated schedule corresponds exactly the cosine schedule, which was developed without any theoretical foundation but is the current state-of-the-art noise schedule.

翻译：本短文旨在证明，去噪扩散概率模型（DDPM）作为一种非齐次离散时间马尔可夫过程，可被表示为在非均匀采样离散时间点上观测到的时齐连续时间马尔可夫过程。令人惊讶的是，该连续时间马尔可夫过程即为广为人知且被充分研究的Ornstein-Uhlenbeck（OU）过程，该过程于20世纪30年代为研究简谐势场中的布朗粒子而提出。我们通过OU过程的解析解建立了DDPM与其之间的形式等价性。进一步证明，非齐次DDPM的噪声调度器设计问题等价于为OU过程设计观测时间点。基于自方差和Fisher信息等具有理论依据的量，我们提出了若干启发式观测时间设计方案，并将其与DDPM中经验性的噪声调度相关联。有趣的是，我们发现基于Fisher信息优化的调度恰好对应于余弦调度——该调度虽缺乏理论根基却已成为当前最优的噪声调度方案。