In this paper, we address the limitations of existing text-to-image diffusion models in generating demographically fair results when given human-related descriptions. These models often struggle to disentangle the target language context from sociocultural biases, resulting in biased image generation. To overcome this challenge, we propose Fair Mapping, a general, model-agnostic, and lightweight approach that modifies a pre-trained text-to-image model by controlling the prompt to achieve fair image generation. One key advantage of our approach is its high efficiency. The training process only requires updating a small number of parameters in an additional linear mapping network. This not only reduces the computational cost but also accelerates the optimization process. We first demonstrate the issue of bias in generated results caused by language biases in text-guided diffusion models. By developing a mapping network that projects language embeddings into an unbiased space, we enable the generation of relatively balanced demographic results based on a keyword specified in the prompt. With comprehensive experiments on face image generation, we show that our method significantly improves image generation performance when prompted with descriptions related to human faces. By effectively addressing the issue of bias, we produce more fair and diverse image outputs. This work contributes to the field of text-to-image generation by enhancing the ability to generate images that accurately reflect the intended demographic characteristics specified in the text.

翻译：本文针对现有文本到图像扩散模型在生成与人类描述相关的图像时无法产生人口统计学公平结果的问题，提出了一种通用、模型无关且轻量级的公平映射方法。该方法通过控制提示词对预训练文本到图像模型进行修正，从而实现公平的图像生成。其关键优势在于高效性：训练过程仅需更新额外线性映射网络中的少量参数，这既降低了计算成本，又加速了优化进程。我们首先揭示了文本引导扩散模型中因语言偏差导致的生成结果偏见问题，通过构建将语言嵌入投影到无偏空间的映射网络，实现了基于提示词中指定关键词的相对均衡人口统计结果生成。基于人脸图像生成的综合实验表明，当使用与人类面部描述相关的提示词时，本方法显著提升了图像生成性能。通过有效处理偏见问题，我们生成了更公平且多样化的图像输出。该工作通过增强生成准确反映文本中预期人口统计特征图像的能力，为文本到图像生成领域做出了贡献。