Probabilistic generative models based on measure transport, such as diffusion and flow-based models, are often formulated in the language of Markovian stochastic dynamics, where the choice of the underlying process impacts both algorithmic design choices and theoretical analysis. In this paper, we aim to establish a rigorous mathematical foundation for denoising Markov models, a broad class of generative models that postulate a forward process transitioning from the target distribution to a simple, easy-to-sample distribution, alongside a backward process particularly constructed to enable efficient sampling in the reverse direction. Leveraging deep connections with nonequilibrium statistical mechanics and generalized Doob's $h$-transform, we propose a minimal set of assumptions that ensure: (1) explicit construction of the backward generator, (2) a unified variational objective directly minimizing the measure transport discrepancy, and (3) adaptations of the classical score-matching approach across diverse dynamics. Our framework unifies existing formulations of continuous and discrete diffusion models, identifies the most general form of denoising Markov models under certain regularity assumptions on forward generators, and provides a systematic recipe for designing denoising Markov models driven by arbitrary L\'evy-type processes. We illustrate the versatility and practical effectiveness of our approach through novel denoising Markov models employing geometric Brownian motion and jump processes as forward dynamics, highlighting the framework's potential flexibility and capability in modeling complex distributions.
翻译:基于测度传输的概率生成模型,如扩散模型和基于流的模型,通常以马尔可夫随机动力学的语言表述,其中底层过程的选择同时影响算法设计决策和理论分析。本文旨在为去噪马尔可夫模型建立一个严格的数学基础,这是一类广泛的生成模型,其假设一个从目标分布过渡到简单、易于采样分布的前向过程,以及一个特别构造的、以实现高效反向采样的后向过程。利用与非平衡统计力学和广义Doob $h$-变换的深刻联系,我们提出了一组最小假设,以确保:(1)后向生成元的显式构造,(2)直接最小化测度传输差异的统一变分目标,以及(3)经典得分匹配方法在不同动力学中的适应性。我们的框架统一了连续和离散扩散模型的现有表述,在关于前向生成元的某些正则性假设下,识别出去噪马尔可夫模型的最一般形式,并为设计由任意Lévy型过程驱动的去噪马尔可夫模型提供了系统化方案。我们通过采用几何布朗运动和跳跃过程作为前向动力学的新型去噪马尔可夫模型,展示了我们方法的普适性和实际有效性,凸显了该框架在建模复杂分布方面的潜在灵活性和能力。