Recent large-scale T2I models like DALLE-3 have made progress on improving fairness in single-subject generation, i.e. generating a one-person image. However, we reveal that these improved models still demonstrate considerable biases when simply generating two people. To systematically evaluate T2I models in this challenging generation setting, we propose the Paired Stereotype Test (PST) framework, established as a dual-subject generation task, i.e. generating two people in the same image. The setting in PST is especially challenging, as the two individuals are described with social identities that are male-stereotyped and female-stereotyped, respectively, e.g. "a CEO" and "an Assistant". It is easy for T2I models to unfairly follow gender stereotypes in this contrastive setting. We establish a metric, Stereotype Score (SS), to quantitatively measure the adherence to gender stereotypes in generated images. Using PST, we evaluate two aspects of gender biases in DALLE-3 -- the widely-identified bias in gendered occupation, as well as a novel aspect: bias in organizational power. Results show that despite generating seemingly fair or even anti-stereotype single-person images, DALLE-3 still shows notable biases under PST -- for instance, in experiments on gender-occupational stereotypes, over 74% model generations demonstrate biases. Moreover, compared to single-person settings, DALLE-3 is more likely to perpetuate male-associated stereotypes under PST. Our work pioneers the research on bias in dual-subject generation, and our proposed PST framework can be easily extended for further experiments, establishing a valuable contribution.

翻译：近期的大规模文本到图像（T2I）模型（如DALLE-3）在提升单主体生成（即生成单人图像）的公平性方面取得了进展。然而，我们发现这些改进后的模型在简单生成两人图像时仍表现出显著的偏见。为系统评估T2I模型在这一具有挑战性的生成场景中的表现，我们提出了成对刻板印象测试（PST）框架，该框架构建为双主体生成任务，即在同一图像中生成两个人。PST的设置尤为困难，因为两个个体分别被描述为具有男性刻板印象和女性刻板印象的社会身份，例如“一位CEO”和“一位助理”。在这种对比性设置下，T2I模型很容易不公平地遵循性别刻板印象。我们建立了一个度量标准——刻板印象分数（SS），以量化衡量生成图像中对性别刻板印象的遵循程度。利用PST，我们评估了DALLE-3中性别偏见的两个方面：广泛认知的职业性别偏见，以及一个新方面：组织权力中的偏见。结果表明，尽管DALLE-3能生成看似公平甚至反刻板印象的单人图像，但在PST下仍显示出明显的偏见——例如，在性别-职业刻板印象实验中，超过74%的模型生成结果表现出偏见。此外，与单人生成设置相比，DALLE-3在PST下更倾向于延续与男性相关的刻板印象。我们的工作开创了双主体生成中偏见研究的先河，所提出的PST框架可轻松扩展用于进一步实验，具有重要的学术价值。