Text-to-image models are known to propagate social biases. For example, when prompted to generate images of people in certain professions, these models tend to systematically generate specific genders or ethnicities. In this paper, we show that this bias is already present in the text encoder of the model and introduce a Mixture-of-Experts approach by identifying text-encoded bias in the latent space and then creating a Bias-Identification Gate mechanism. More specifically, we propose MoESD (Mixture of Experts Stable Diffusion) with BiAs (Bias Adapters) to mitigate gender bias in text-to-image models. We also demonstrate that introducing an arbitrary special token to the prompt is essential during the mitigation process. With experiments focusing on gender bias, we show that our approach successfully mitigates gender bias while maintaining image quality.

翻译：文本到图像模型已知会传播社会偏见。例如，当提示生成特定职业的人物图像时，这些模型往往系统性地生成特定性别或种族的图像。本文证明，这种偏见已存在于模型的文本编码器中，并提出一种专家混合方法：首先在潜在空间中识别文本编码的偏见，随后构建偏见识别门控机制。具体而言，我们提出配备偏见适配器（BiAs）的MoESD（专家混合稳定扩散）模型，以缓解文本到图像模型中的性别偏见。我们还证明，在缓解过程中向提示词引入任意特殊标记至关重要。通过针对性别偏见的实验，我们表明该方法在保持图像质量的同时，成功缓解了性别偏见。