Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming, and there exists a noticeable gap in sample quality between EBMs and other generative frameworks like GANs and diffusion models. To close this gap, inspired by the recent efforts of learning EBMs by maximizing diffusion recovery likelihood (DRL), we propose cooperative diffusion recovery likelihood (CDRL), an effective approach to tractably learn and sample from a series of EBMs defined on increasingly noisy versions of a dataset, paired with an initializer model for each EBM. At each noise level, the two models are jointly estimated within a cooperative training framework: samples from the initializer serve as starting points that are refined by a few MCMC sampling steps from the EBM. The EBM is then optimized by maximizing recovery likelihood, while the initializer model is optimized by learning from the difference between the refined samples and the initial samples. In addition, we made several practical designs for EBM training to further improve the sample quality. Combining these advances, our approach significantly boost the generation performance compared to existing EBM methods on CIFAR-10 and ImageNet datasets. We also demonstrate the effectiveness of our models for several downstream tasks, including classifier-free guided generation, compositional generation, image inpainting and out-of-distribution detection.

翻译：在高维数据上训练能量模型（EBMs）既具挑战性又耗时，且EBMs与生成对抗网络（GANs）及扩散模型等其他生成框架之间存在显著的样本质量差距。为弥合这一差距，受近期通过最大化扩散恢复似然（DRL）学习EBMs研究的启发，我们提出合作扩散恢复似然（CDRL）方法——一种能够有效学习并采样一系列定义在数据集逐渐加噪版本上的EBMs的高效方法，并为每个EBM配备一个初始模型。在每个噪声水平上，两个模型在合作训练框架内联合估计：初始模型生成的样本作为起点，经过EBM的少量MCMC采样步骤优化后得到精炼样本。随后，通过最大化恢复似然来优化EBM，而初始模型则通过从精炼样本与初始样本的差异中学习进行优化。此外，我们针对EBM训练进行了多项实用设计以进一步提升样本质量。结合这些改进，我们的方法在CIFAR-10和ImageNet数据集上相较于现有EBM方法显著提升了生成性能。我们还在多个下游任务中验证了模型的有效性，包括无分类器引导生成、组合生成、图像修复及分布外检测。