The rapidly evolving fields of e-commerce and metaverse continue to seek innovative approaches to enhance the consumer experience. At the same time, recent advancements in the development of diffusion models have enabled generative networks to create remarkably realistic images. In this context, image-based virtual try-on, which consists in generating a novel image of a target model wearing a given in-shop garment, has yet to capitalize on the potential of these powerful generative solutions. This work introduces LaDI-VTON, the first Latent Diffusion textual Inversion-enhanced model for the Virtual Try-ON task. The proposed architecture relies on a latent diffusion model extended with a novel additional autoencoder module that exploits learnable skip connections to enhance the generation process preserving the model's characteristics. To effectively maintain the texture and details of the in-shop garment, we propose a textual inversion component that can map the visual features of the garment to the CLIP token embedding space and thus generate a set of pseudo-word token embeddings capable of conditioning the generation process. Experimental results on Dress Code and VITON-HD datasets demonstrate that our approach outperforms the competitors by a consistent margin, achieving a significant milestone for the task. Source code and trained models will be publicly released at: https://github.com/miccunifi/ladi-vton.

翻译：电子商务和元宇宙领域的快速发展持续寻求创新方法来提升消费者体验。与此同时，扩散模型的最新进展使得生成网络能够创造出高度逼真的图像。在此背景下，基于图像的虚拟试穿——即生成目标模特穿着指定店内服装的新图像——尚未充分利用这些强大生成式解决方案的潜力。本文提出LaDI-VTON，这是首个用于虚拟试穿任务的潜在扩散文本反演增强模型。所提出的架构基于一个潜在扩散模型，并扩展了一个新颖的附加自编码器模块，该模块利用可学习的跳跃连接来增强生成过程，同时保留模特特性。为了有效保持店内服装的纹理和细节，我们提出一个文本反演组件，能够将服装的视觉特征映射到CLIP令牌嵌入空间，从而生成一组能够条件化生成过程的伪词令牌嵌入。在Dress Code和VITON-HD数据集上的实验结果表明，我们的方法以一致的性能优势超越现有竞争对手，为该任务实现了重要里程碑。源代码和训练模型将在https://github.com/miccunifi/ladi-vton公开提供。