A persistent challenge in conditional image synthesis has been to generate diverse output images from the same input image despite only one output image being observed per input image. GAN-based methods are prone to mode collapse, which leads to low diversity. To get around this, we leverage Implicit Maximum Likelihood Estimation (IMLE) which can overcome mode collapse fundamentally. IMLE uses the same generator as GANs but trains it with a different, non-adversarial objective which ensures each observed image has a generated sample nearby. Unfortunately, to generate high-fidelity images, prior IMLE-based methods require a large number of samples, which is expensive. In this paper, we propose a new method to get around this limitation, which we dub Conditional Hierarchical IMLE (CHIMLE), which can generate high-fidelity images without requiring many samples. We show CHIMLE significantly outperforms the prior best IMLE, GAN and diffusion-based methods in terms of image fidelity and mode coverage across four tasks, namely night-to-day, 16x single image super-resolution, image colourization and image decompression. Quantitatively, our method improves Fr\'echet Inception Distance (FID) by 36.9% on average compared to the prior best IMLE-based method, and by 27.5% on average compared to the best non-IMLE-based general-purpose methods.

翻译：在条件图像合成中，一个长期存在的挑战是：尽管每个输入图像仅对应一个观测到的输出图像，但仍需从同一输入生成多样化的输出图像。基于生成对抗网络（GAN）的方法容易陷入模式坍塌，导致多样性不足。为解决这一问题，我们利用隐式最大似然估计（IMLE），该方法可从根本上克服模式坍塌。IMLE采用与GAN相同的生成器，但通过非对抗性目标函数进行训练，确保每个观测图像附近都存在生成的样本。然而，为生成高保真图像，现有基于IMLE的方法需要大量采样，这导致计算成本高昂。本文提出一种克服该局限的新方法——条件层次化IMLE（CHIMLE），该方法无需大量采样即可生成高保真图像。在夜间转白天、16倍单图像超分辨率、图像着色和图像解压缩四项任务中，CHIMLE在图像保真度和模式覆盖方面显著优于此前最优的基于IMLE、GAN及扩散模型的方法。定量分析表明，与先前最优的基于IMLE的方法相比，本方法将弗雷歇初始距离（FID）平均降低36.9%；与最优的非IMLE通用方法相比，平均降低27.5%。