Large text-to-image models have revolutionized the ability to generate imagery using natural language. However, particularly unique or personal visual concepts, such as pets and furniture, will not be captured by the original model. This has led to interest in how to personalize a text-to-image model. Despite significant progress, this task remains a formidable challenge, particularly in preserving the subject's identity. Most researchers attempt to address this issue by modifying model architectures. These methods are capable of keeping the subject structure and color but fail to preserve identity details. Towards this issue, our approach takes a data-centric perspective. We introduce a novel regularization dataset generation strategy on both the text and image level. This strategy enables the model to preserve fine details of the desired subjects, such as text and logos. Our method is architecture-agnostic and can be flexibly applied on various text-to-image models. We show on established benchmarks that our data-centric approach forms the new state of the art in terms of identity preservation and text alignment.

翻译：大规模文本到图像模型革新了使用自然语言生成图像的能力。然而，原始模型无法捕捉到宠物、家具等独特或个性化的视觉概念。这引发了如何个性化文本到图像模型的研究兴趣。尽管取得了显著进展，但该任务仍面临严峻挑战，尤其是在保持主体身份方面。多数研究者试图通过修改模型架构来解决这一问题。这些方法能够保持主体结构和颜色，但未能保留身份细节。针对此问题，我们的方法从数据中心的视角出发。我们在文本和图像层面引入了一种新颖的正则化数据集生成策略。该策略使模型能够保留所需主体的精细细节，如文字和标识。我们的方法具有架构无关性，可灵活应用于多种文本到图像模型。我们在现有基准测试上证明，这种数据中心方法在身份保持和文本对齐方面达到了新的最优水平。