Bias amplification is a phenomenon in which models increase imbalances present in the training data. In this paper, we study bias amplification in the text-to-image domain using Stable Diffusion by comparing gender ratios in training vs. generated images. We find that the model appears to amplify gender-occupation biases found in the training data (LAION). However, we discover that amplification can largely be attributed to discrepancies between training captions and model prompts. For example, an inherent difference is that captions from the training data often contain explicit gender information while the prompts we use do not, which leads to a distribution shift and consequently impacts bias measures. Once we account for various distributional differences between texts used for training and generation, we observe that amplification decreases considerably. Our findings illustrate the challenges of comparing biases in models and the data they are trained on, and highlight confounding factors that contribute to bias amplification.

翻译：偏差放大是指模型加剧训练数据中不平衡现象的现象。本文通过比较训练图像与生成图像中的性别比例，利用稳定扩散模型研究文本到图像领域中的偏差放大问题。研究发现，该模型似乎放大了训练数据（LAION）中存在的性别-职业偏见。然而，我们注意到这种放大在很大程度上可归因于训练文本描述与模型提示词之间的差异。例如，训练数据中的文本描述通常包含明确的性别信息，而我们所使用的提示词则没有，这导致了分布偏移，进而影响偏差度量。一旦我们考虑到训练和生成所用文本之间的各类分布差异，即可观察到偏差放大现象显著减弱。我们的研究结果揭示了比较模型及其训练数据偏差的挑战性，并凸显了促成偏差放大的混杂因素。