We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function. We benchmark several diffusion-structured inference methods, including simulation-based variational approaches and off-policy methods (continuous generative flow networks). Our results shed light on the relative advantages of existing algorithms while bringing into question some claims from past work. We also propose a novel exploration strategy for off-policy methods, based on local search in the target space with the use of a replay buffer, and show that it improves the quality of samples on a variety of target distributions. Our code for the sampling methods and benchmarks studied is made public at https://github.com/GFNOrg/gfn-diffusion as a base for future work on diffusion models for amortized inference.

翻译：我们研究了训练扩散模型从具有给定未归一化密度或能量函数的分布中采样的问题。我们对几种扩散结构推理方法进行了基准测试，包括基于模拟的变分方法和离策略方法（连续生成流网络）。我们的结果揭示了现有算法的相对优势，同时质疑了以往工作中的一些主张。我们还提出了一种新的离策略方法探索策略，该方法基于在目标空间中利用重放缓冲区进行局部搜索，并证明它能在多种目标分布上提高样本质量。我们将在https://github.com/GFNOrg/gfn-diffusion上公开发布采样方法和基准测试的代码，作为未来关于摊销推理扩散模型研究的基础。