Deep learning based visual-linguistic multimodal models such as Contrastive Language Image Pre-training (CLIP) have become increasingly popular recently and are used within text-to-image generative models such as DALL-E and Stable Diffusion. However, gender and other social biases have been uncovered in these models, and this has the potential to be amplified and perpetuated through AI systems. In this paper, we present a methodology for auditing multimodal models that consider gender, informed by concepts from transnational feminism, including regional and cultural dimensions. Focusing on CLIP, we found evidence of significant gender bias with varying patterns across global regions. Harmful stereotypical associations were also uncovered related to visual cultural cues and labels such as terrorism. Levels of gender bias uncovered within CLIP for different regions aligned with global indices of societal gender equality, with those from the Global South reflecting the highest levels of gender bias.

翻译：基于深度学习的视觉-语言多模态模型（如对比语言-图像预训练模型CLIP）近年来日益流行，并被应用于DALL-E和Stable Diffusion等文本到图像生成模型中。然而，这些模型已被发现存在性别及其他社会偏见，且此类偏见可能会通过人工智能系统被放大和固化。本文提出了一种基于跨国女性主义理论（涵盖地域与文化维度）的审计方法，用于评估多模态模型中的性别偏见问题。以CLIP模型为研究对象，我们发现其存在显著的性别偏见，且在不同全球区域呈现差异模式。研究还揭示了与视觉文化线索及"恐怖主义"等标签相关的有害刻板联想。CLIP模型在不同区域显现的性别偏见水平与全球社会性别平等指数高度吻合，其中来自全球南方地区的模型表现出最严重的性别偏见程度。