Text-to-image generative models often reflect the biases of the training data, leading to unequal representations of underrepresented groups. This study investigates inclusive text-to-image generative models that generate images based on human-written prompts and ensure the resulting images are uniformly distributed across attributes of interest. Unfortunately, directly expressing the desired attributes in the prompt often leads to sub-optimal results due to linguistic ambiguity or model misrepresentation. Hence, this paper proposes a drastically different approach that adheres to the maxim that "a picture is worth a thousand words". We show that, for some attributes, images can represent concepts more expressively than text. For instance, categories of skin tones are typically hard to specify by text but can be easily represented by example images. Building upon these insights, we propose a novel approach, ITI-GEN, that leverages readily available reference images for Inclusive Text-to-Image GENeration. The key idea is learning a set of prompt embeddings to generate images that can effectively represent all desired attribute categories. More importantly, ITI-GEN requires no model fine-tuning, making it computationally efficient to augment existing text-to-image models. Extensive experiments demonstrate that ITI-GEN largely improves over state-of-the-art models to generate inclusive images from a prompt. Project page: https://czhang0528.github.io/iti-gen.

翻译：文本到图像生成模型常常反映训练数据中的偏见，导致对代表性不足群体的不平等呈现。本研究旨在探索包容性文本到图像生成模型，该模型基于人类编写的提示生成图像，并确保生成的图像在所关注的属性上均匀分布。然而，直接在提示中表达所需属性常因语言歧义或模型误表示而导致次优结果。为此，本文提出了一种截然不同的方法，遵循“一图胜千言”的原则。我们证明，对于某些属性，图像比文本更能富有表现力地呈现概念。例如，肤色类别通常难以通过文本精确指定，但能通过示例图像轻松表示。基于这些见解，我们提出了一种新方法ITI-GEN（Inclusive Text-to-Image GENeration），利用现成的参考图像实现包容性文本到图像生成。其核心思想是学习一组提示嵌入，以生成能够有效表示所有期望属性类别的图像。更重要的是，ITI-GEN无需模型微调，因此计算高效，能够增强现有文本到图像模型。大量实验表明，ITI-GEN在通过提示生成包容性图像方面显著优于现有最先进模型。项目页面：https://czhang0528.github.io/iti-gen。