Text-to-image diffusion models trained on a fixed set of resolutions often fail to generalize, even when asked to generate images at lower resolutions than those seen during training. High-resolution text-to-image generators are currently unable to easily offer an out-of-the-box budget-efficient alternative to their users who might not need high-resolution images. We identify a key technical insight in diffusion models that when addressed can help tackle this limitation: Noise schedulers have unequal perceptual effects across resolutions. The same level of noise removes disproportionately more signal from lower-resolution images than from high-resolution images, leading to a train-test mismatch. We propose NoiseShift, a training-free method that recalibrates the noise level of the denoiser conditioned on resolution size. NoiseShift requires no changes to model architecture or sampling schedule and is compatible with existing models. When applied to Stable Diffusion 3, Stable Diffusion 3.5, and Flux-Dev, quality at low resolutions is significantly improved. On LAION-COCO, NoiseShift improves SD3.5 by 15.89%, SD3 by 8.56%, and Flux-Dev by 2.44% in FID on average. On CelebA, NoiseShift improves SD3.5 by 10.36%, SD3 by 5.19%, and Flux-Dev by 3.02% in FID on average. These results demonstrate the effectiveness of NoiseShift in mitigating resolution-dependent artifacts and enhancing the quality of low-resolution image generation.

翻译：在固定分辨率集上训练的文本到图像扩散模型往往难以泛化，即使要求生成低于训练所见分辨率的图像时也是如此。当前的高分辨率文本到图像生成器无法轻易为其可能不需要高分辨率图像的用户提供开箱即用的经济高效替代方案。我们在扩散模型中识别出一个关键的技术洞见，解决该问题有助于克服此限制：噪声调度器在不同分辨率上具有不均衡的感知效应。相同水平的噪声从低分辨率图像中移除的信号量不成比例地高于高分辨率图像，导致训练-测试失配。我们提出NoiseShift，这是一种无需训练的方法，可根据分辨率大小对去噪器的噪声水平进行重校准。NoiseShift无需改变模型架构或采样计划，且与现有模型兼容。当应用于Stable Diffusion 3、Stable Diffusion 3.5和Flux-Dev时，低分辨率下的图像质量得到显著提升。在LAION-COCO数据集上，NoiseShift将SD3.5的FID平均提升15.89%，SD3提升8.56%，Flux-Dev提升2.44%。在CelebA数据集上，NoiseShift将SD3.5的FID平均提升10.36%，SD3提升5.19%，Flux-Dev提升3.02%。这些结果证明了NoiseShift在缓解分辨率相关伪影和提升低分辨率图像生成质量方面的有效性。