Generative multimodal models based on diffusion models have seen tremendous growth and advances in recent years. Models such as DALL-E and Stable Diffusion have become increasingly popular and successful at creating images from texts, often combining abstract ideas. However, like other deep learning models, they also reflect social biases they inherit from their training data, which is often crawled from the internet. Manually auditing models for biases can be very time and resource consuming and is further complicated by the unbounded and unconstrained nature of inputs these models can take. Research into bias measurement and quantification has generally focused on small single-stage models working on a single modality. Thus the emergence of multistage multimodal models requires a different approach. In this paper, we propose Multimodal Composite Association Score (MCAS) as a new method of measuring gender bias in multimodal generative models. Evaluating both DALL-E 2 and Stable Diffusion using this approach uncovered the presence of gendered associations of concepts embedded within the models. We propose MCAS as an accessible and scalable method of quantifying potential bias for models with different modalities and a range of potential biases.

翻译：基于扩散模型的生成式多模态模型近年来取得了巨大的进步与发展。诸如DALL-E和Stable Diffusion等模型在从文本生成图像（往往结合抽象概念）方面日益流行且表现优异。然而，与其他深度学习模型类似，这些模型也反映了其训练数据（通常来自互联网抓取）所继承的社会偏见。人工审计模型偏见不仅耗时耗资源，而且由于模型能接受无界且无约束的输入，这一过程更加复杂化。针对偏见的测量与量化研究通常聚焦于处理单一模态的小型单阶段模型。因此，多阶段多模态模型的出现需要采用不同的方法。本文提出多模态复合关联评分（MCAS），作为衡量多模态生成模型中性别偏见的新方法。通过使用该方法评估DALL-E 2和Stable Diffusion，我们发现了模型中嵌入的概念的性别关联表征。我们提出MCAS作为一种可访问且可扩展的方法，用于量化不同模态模型及多种潜在偏见的可能性。