Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming, and there exists a noticeable gap in sample quality between EBMs and other generative frameworks like GANs and diffusion models. To close this gap, inspired by the recent efforts of learning EBMs by maximizing diffusion recovery likelihood (DRL), we propose cooperative diffusion recovery likelihood (CDRL), an effective approach to tractably learn and sample from a series of EBMs defined on increasingly noisy versions of a dataset, paired with an initializer model for each EBM. At each noise level, the two models are jointly estimated within a cooperative training framework: samples from the initializer serve as starting points that are refined by a few MCMC sampling steps from the EBM. The EBM is then optimized by maximizing recovery likelihood, while the initializer model is optimized by learning from the difference between the refined samples and the initial samples. In addition, we made several practical designs for EBM training to further improve the sample quality. Combining these advances, our approach significantly boost the generation performance compared to existing EBM methods on CIFAR-10 and ImageNet datasets. We also demonstrate the effectiveness of our models for several downstream tasks, including classifier-free guided generation, compositional generation, image inpainting and out-of-distribution detection.

翻译：在高维数据上训练能量模型（EBM）既具有挑战性又耗时，且EBM与生成对抗网络（GAN）、扩散模型等其他生成框架之间存在显著的样本质量差距。为弥补这一差距，受近期通过最大化扩散恢复似然（DRL）学习EBM工作的启发，我们提出合作扩散恢复似然（CDRL），这是一种高效方法，用于可解地学习并采样一系列定义在数据集逐步加噪版本上的EBM，并为每个EBM配对一个初始化模型。在每个噪声水平上，两个模型在合作训练框架内联合估计：初始化模型的样本作为起始点，经过EBM的少量MCMC采样步骤精炼。随后通过最大化恢复似然优化EBM，同时初始化模型通过从精炼样本与初始样本的差异中学习进行优化。此外，我们为EBM训练设计了若干实用方案以进一步提升样本质量。结合这些改进，我们的方法在CIFAR-10和ImageNet数据集上显著提升了生成性能，相较于现有EBM方法。我们还展示了模型在多个下游任务中的有效性，包括无分类器引导生成、组合生成、图像修复及分布外检测。