Recent large-scale Text-To-Image (T2I) models such as DALLE-3 demonstrate great potential in new applications, but also face unprecedented fairness challenges. Prior studies revealed gender biases in single-person image generation, but T2I model applications might require portraying two or more people simultaneously. Potential biases in this setting remain unexplored, leading to fairness-related risks in usage. To study these underlying facets of gender biases in T2I models, we propose a novel Paired Stereotype Test (PST) bias evaluation framework. PST prompts the model to generate two individuals in the same image. They are described with two social identities that are stereotypically associated with the opposite gender. Biases can then be measured by the level of conformation to gender stereotypes in generated images. Using PST, we evaluate DALLE-3 from 2 perspectives: biases in gendered occupation and biases in organizational power. Despite seemingly fair or even anti-stereotype single-person generations, PST still unveils gendered occupational and power associations. Moreover, compared to single-person settings, DALLE-3 generates noticeably more masculine figures under PST for individuals with male-stereotypical identities. PST is therefore effective in revealing underlying gender biases in DALLE-3 that single-person settings cannot capture. Our findings reveal the complicated patterns of gender biases in modern T2I models, further highlighting the critical fairness challenges in multimodal generative systems.

翻译：近期诸如DALLE-3等大规模文本到图像（T2I）模型在新应用领域展现出巨大潜力，但也面临前所未有的公平性挑战。先前研究揭示了单人生成图像中的性别偏见，但T2I模型的应用可能需要同时描绘两人或更多人。该情境下的潜在偏见尚未得到探索，导致使用时存在公平性相关风险。为研究T2I模型中性别偏见的这些潜在层面，我们提出了一种新颖的配对刻板印象测试（PST）偏见评估框架。PST引导模型在同一图像中生成两个个体，这两个个体分别具有刻板印象中与相反性别相关联的两种社会身份。随后可通过生成图像对性别刻板印象的遵从程度来测量偏见。利用PST，我们从两个视角评估DALLE-3：职业性别偏见与组织权力偏见。尽管面对单人生成时看似公平甚至呈现反刻板印象，PST仍揭示了性别化的职业与权力关联。此外，相较于单人设置，DALLE-3在PST下对具有男性刻板身份特征的个体生成了明显更具男性特征的形象。因此，PST能有效揭示DALLE-3中单人设置无法捕捉的潜在性别偏见。我们的研究结果揭示了现代T2I模型中性别偏见的复杂模式，进一步凸显了多模态生成系统中关键的公平性挑战。