We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function. We benchmark several diffusion-structured inference methods, including simulation-based variational approaches and off-policy methods (continuous generative flow networks). Our results shed light on the relative advantages of existing algorithms while bringing into question some claims from past work. We also propose a novel exploration strategy for off-policy methods, based on local search in the target space with the use of a replay buffer, and show that it improves the quality of samples on a variety of target distributions. Our code for the sampling methods and benchmarks studied is made public at https://github.com/GFNOrg/gfn-diffusion as a base for future work on diffusion models for amortized inference.

翻译：我们研究训练扩散模型从具有给定未归一化密度或能量函数的分布中采样的问题。我们对几种扩散结构推断方法进行了基准测试，包括基于模拟的变分方法和离策略方法（连续生成流网络）。我们的研究结果揭示了现有算法的相对优势，同时质疑了以往研究中的一些观点。我们还提出了一种新颖的离策略方法探索策略，该策略基于在目标空间中使用重放缓冲区进行局部搜索，并证明它提高了多种目标分布上的样本质量。我们将所研究的采样方法和基准测试的代码公开发布于 https://github.com/GFNOrg/gfn-diffusion，作为未来关于扩散模型用于摊销推理研究的基础。