Text-to-image diffusion models have shown remarkable success in generating personalized subjects based on a few reference images. However, current methods often fail when generating multiple subjects simultaneously, resulting in mixed identities with combined attributes from different subjects. In this work, we present MuDI, a novel framework that enables multi-subject personalization by effectively decoupling identities from multiple subjects. Our main idea is to utilize segmented subjects generated by a foundation model for segmentation (Segment Anything) for both training and inference, as a form of data augmentation for training and initialization for the generation process. Moreover, we further introduce a new metric to better evaluate the performance of our method on multi-subject personalization. Experimental results show that our MuDI can produce high-quality personalized images without identity mixing, even for highly similar subjects as shown in Figure 1. Specifically, in human evaluation, MuDI obtains twice the success rate for personalizing multiple subjects without identity mixing over existing baselines and is preferred over 70% against the strongest baseline.

翻译：文本到图像扩散模型在基于少量参考图像生成个性化主体方面已展现出显著成功。然而，现有方法在同时生成多个主体时往往失效，导致不同主体的属性混合而产生身份混淆。本研究提出MuDI，一种通过有效解耦多主体身份实现多主体个性化的新型框架。我们的核心思想是利用基础分割模型（Segment Anything）生成的已分割主体，同时用于训练和推理阶段——既作为训练时的数据增强手段，也作为生成过程的初始化依据。此外，我们进一步引入新的评估指标以更准确衡量多主体个性化任务的性能。实验结果表明，即使对于如图1所示的高度相似主体，我们的MuDI仍能生成无身份混淆的高质量个性化图像。具体而言，在人工评估中，MuDI实现多主体个性化且无身份混淆的成功率是现有基准方法的两倍，并在与最强基线的对比中获得超过70%的偏好度。