There is a bias in the inference pipeline of most diffusion models. This bias arises from a signal leak whose distribution deviates from the noise distribution, creating a discrepancy between training and inference processes. We demonstrate that this signal-leak bias is particularly significant when models are tuned to a specific style, causing sub-optimal style matching. Recent research tries to avoid the signal leakage during training. We instead show how we can exploit this signal-leak bias in existing diffusion models to allow more control over the generated images. This enables us to generate images with more varied brightness, and images that better match a desired style or color. By modeling the distribution of the signal leak in the spatial frequency and pixel domains, and including a signal leak in the initial latent, we generate images that better match expected results without any additional training.

翻译：大多数扩散模型的推理流程中存在一种偏差。这种偏差源于信号泄露，其分布偏离了噪声分布，导致训练与推理过程不一致。我们证明，当模型针对特定风格进行微调时，该信号泄露偏差尤为显著，会造成次优的风格匹配效果。近期研究试图在训练中避免信号泄露。我们则展示了如何利用现有扩散模型中的这种信号泄露偏差，从而增强对生成图像的控制能力。这使我们能够生成亮度更多样化、且更贴合期望风格或色彩的图像。通过在空间频率域和像素域中对信号泄露分布进行建模，并在初始潜变量中包含信号泄露，我们无需额外训练即可生成更符合预期结果的图像。