We introduce generator matching, a modality-agnostic framework for generative modeling using arbitrary Markov processes. Generators characterize the infinitesimal evolution of a Markov process, which we leverage for generative modeling in a similar vein to flow matching: we construct conditional generators which generate single data points, then learn to approximate the marginal generator which generates the full data distribution. We show that generator matching unifies various generative modeling methods, including diffusion models, flow matching and discrete diffusion models. Furthermore, it provides the foundation to expand the design space to new and unexplored Markov processes such as jump processes. Finally, generator matching enables the construction of superpositions of Markov generative processes and enables the construction of multimodal models in a rigorous manner. We empirically validate our method on protein and image structure generation, showing that superposition with a jump process improves image generation.

翻译：我们提出了生成器匹配，这是一种与模态无关的生成式建模框架，它利用任意马尔可夫过程进行建模。生成器刻画了马尔可夫过程的无穷小演化，我们以类似于流匹配的方式将其用于生成式建模：我们构建生成单个数据点的条件生成器，然后学习逼近生成完整数据分布的边际生成器。我们证明了生成器匹配统一了多种生成式建模方法，包括扩散模型、流匹配和离散扩散模型。此外，它为将设计空间扩展到新的、尚未探索的马尔可夫过程（如跳跃过程）奠定了基础。最后，生成器匹配使得构建马尔可夫生成过程的叠加成为可能，并能以严格的方式构建多模态模型。我们在蛋白质和图像结构生成任务上实证验证了我们的方法，结果表明与跳跃过程的叠加能提升图像生成质量。