In this study, we delve into the generation of high-resolution images from pre-trained diffusion models, addressing persistent challenges, such as repetitive patterns and structural distortions, that emerge when models are applied beyond their trained resolutions. To address this issue, we introduce an innovative, training-free approach FouriScale from the perspective of frequency domain analysis. We replace the original convolutional layers in pre-trained diffusion models by incorporating a dilation technique along with a low-pass operation, intending to achieve structural consistency and scale consistency across resolutions, respectively. Further enhanced by a padding-then-crop strategy, our method can flexibly handle text-to-image generation of various aspect ratios. By using the FouriScale as guidance, our method successfully balances the structural integrity and fidelity of generated images, achieving an astonishing capacity of arbitrary-size, high-resolution, and high-quality generation. With its simplicity and compatibility, our method can provide valuable insights for future explorations into the synthesis of ultra-high-resolution images. The code will be released at https://github.com/LeonHLJ/FouriScale.

翻译：本研究深入探究了基于预训练扩散模型生成高分辨率图像的过程，着力解决模型在超出其训练分辨率应用时出现的重复图案与结构失真等持续挑战。针对此问题，我们从频域分析角度提出了一种创新的免训练方法FouriScale。该方法通过引入膨胀技术与低通操作来替换预训练扩散模型中的原始卷积层，旨在分别实现跨分辨率的结构一致性与尺度一致性。经填充后裁剪策略进一步优化，我们的方法可灵活处理不同纵横比的文本到图像生成任务。借助FouriScale的引导作用，该方法成功平衡了生成图像的结构完整性与保真度，展现出任意尺寸、高分辨率及高品质生成的惊人能力。凭借其简洁性与兼容性，本研究可为未来超高分辨率图像合成的探索提供重要启示。相关代码将发布于https://github.com/LeonHLJ/FouriScale。