We present BootComp, a novel framework based on text-to-image diffusion models for controllable human image generation with multiple reference garments. Here, the main bottleneck is data acquisition for training: collecting a large-scale dataset of high-quality reference garment images per human subject is quite challenging, i.e., ideally, one needs to manually gather every single garment photograph worn by each human. To address this, we propose a data generation pipeline to construct a large synthetic dataset, consisting of human and multiple-garment pairs, by introducing a model to extract any reference garment images from each human image. To ensure data quality, we also propose a filtering strategy to remove undesirable generated data based on measuring perceptual similarities between the garment presented in human image and extracted garment. Finally, by utilizing the constructed synthetic dataset, we train a diffusion model having two parallel denoising paths that use multiple garment images as conditions to generate human images while preserving their fine-grained details. We further show the wide-applicability of our framework by adapting it to different types of reference-based generation in the fashion domain, including virtual try-on, and controllable human image generation with other conditions, e.g., pose, face, etc.

翻译：我们提出了BootComp，一种基于文本到图像扩散模型的新型框架，用于通过多件参考服装实现可控人体图像生成。此处的核心瓶颈在于训练数据的获取：为每个个体收集大规模高质量参考服装图像数据集极具挑战性，即理想情况下需要人工收集每位个体穿着的每一件服装照片。为解决此问题，我们提出一种数据生成流水线，通过引入一个能从每张人体图像中提取任意参考服装图像的模型，构建包含人体与多件服装配对的大规模合成数据集。为确保数据质量，我们还提出一种过滤策略，通过度量人体图像中呈现的服装与提取服装之间的感知相似度来剔除不良生成数据。最终，利用构建的合成数据集，我们训练了一个具有两条并行去噪路径的扩散模型，该模型以多件服装图像为条件生成人体图像，同时保留其细粒度细节。我们进一步展示了该框架的广泛适用性，将其适配至时尚领域中不同类型的基于参考的生成任务，包括虚拟试衣，以及结合其他条件（如姿态、面部等）的可控人体图像生成。