Learning-based Text-to-Image (TTI) models like Stable Diffusion have revolutionized the way visual content is generated in various domains. However, recent research has shown that nonnegligible social bias exists in current state-of-the-art TTI systems, which raises important concerns. In this work, we target resolving the social bias in TTI diffusion models. We begin by formalizing the problem setting and use the text descriptions of bias groups to establish an unsafe direction for guiding the diffusion process. Next, we simplify the problem into a weight optimization problem and attempt a Reinforcement solver, Policy Gradient, which shows sub-optimal performance with slow convergence. Further, to overcome limitations, we propose an iterative distribution alignment (IDA) method. Despite its simplicity, we show that IDA shows efficiency and fast convergence in resolving the social bias in TTI diffusion models. Our code will be released.

翻译：基于学习的文本到图像（TTI）模型（如Stable Diffusion）已在多个领域彻底改变了视觉内容的生成方式。然而，近期研究表明，当前最先进的TTI系统中存在不可忽视的社会偏见，这引发了重要关切。本研究旨在解决TTI扩散模型中的社会偏见问题。我们首先对问题设定进行形式化，利用偏见群体的文本描述建立不安全方向以引导扩散过程。随后将问题简化为权重优化问题，并尝试采用强化学习求解器——策略梯度法，但该方法表现出次优性能且收敛缓慢。为克服局限性，我们进一步提出迭代分布对齐（IDA）方法。尽管形式简单，但实验表明IDA在解决TTI扩散模型社会偏见问题上具有高效性和快速收敛性。我们的代码将公开提供。