With the rapid development of Text-to-Image (T2I) models, biases in human image generation against demographic social groups become a significant concern, impacting fairness and ethical standards in AI. Some researchers propose their methods to tackle with the issue. However, existing methods are designed for specific models with fixed prompts, limiting their adaptability to the fast-evolving models and diverse practical scenarios. Moreover, they neglect the impact of hallucinations, leading to discrepancies between expected and actual results. To address these issues, we introduce VersusDebias, a novel and universal debiasing framework for biases in arbitrary T2I models, consisting of an array generation (AG) module and an image generation (IG) module. The self-adaptive AG module generates specialized attribute arrays to post-process hallucinations and debias multiple attributes simultaneously. The IG module employs a small language model to modify prompts according to the arrays and drives the T2I model to generate debiased images, enabling zero-shot debiasing. Extensive experiments demonstrate VersusDebias's capability to debias any models across gender, race, and age simultaneously. In both zero-shot and few-shot scenarios, VersusDebias outperforms existing methods, showcasing its exceptional utility. Our work is accessible at https://github.com/VersusDebias/VersusDebias to ensure reproducibility and facilitate further research.
翻译:随着文本到图像(T2I)模型的快速发展,针对人口统计社会群体的人类图像生成偏见已成为一个重大问题,影响了人工智能的公平性与伦理标准。已有研究者提出多种方法应对此问题。然而,现有方法多针对特定模型且依赖固定提示,难以适应快速迭代的模型与多样化的实际应用场景。此外,这些方法忽视了幻觉效应的影响,导致预期结果与实际生成之间存在偏差。为解决上述问题,我们提出了VersusDebias——一种适用于任意T2I模型的通用去偏框架,该框架由阵列生成(AG)模块与图像生成(IG)模块构成。自适应的AG模块生成专用属性阵列,用于后处理幻觉效应并同时对多属性进行去偏。IG模块采用小型语言模型根据属性阵列修改提示词,驱动T2I模型生成去偏图像,实现零样本去偏。大量实验证明VersusDebias能够同时对任意模型的性别、种族与年龄等多维度偏见进行去偏。在零样本与少样本场景下,VersusDebias均优于现有方法,展现出卓越的实用性。本研究已开源至https://github.com/VersusDebias/VersusDebias,以确保可复现性并促进后续研究。