The rapidly evolving fields of e-commerce and metaverse continue to seek innovative approaches to enhance the consumer experience. At the same time, recent advancements in the development of diffusion models have enabled generative networks to create remarkably realistic images. In this context, image-based virtual try-on, which consists in generating a novel image of a target model wearing a given in-shop garment, has yet to capitalize on the potential of these powerful generative solutions. This work introduces LaDI-VTON, the first Latent Diffusion textual Inversion-enhanced model for the Virtual Try-ON task. The proposed architecture relies on a latent diffusion model extended with a novel additional autoencoder module that exploits learnable skip connections to enhance the generation process preserving the model's characteristics. To effectively maintain the texture and details of the in-shop garment, we propose a textual inversion component that can map the visual features of the garment to the CLIP token embedding space and thus generate a set of pseudo-word token embeddings capable of conditioning the generation process. Experimental results on Dress Code and VITON-HD datasets demonstrate that our approach outperforms the competitors by a consistent margin, achieving a significant milestone for the task. Source code and trained models are publicly available at: https://github.com/miccunifi/ladi-vton.

翻译：电子商务与元宇宙领域正在飞速发展，持续寻求创新方法以提升消费者体验。与此同时，扩散模型的最新进展使得生成网络能够创造出极其逼真的图像。在此背景下，基于图像的虚拟试穿——即生成目标模特穿着特定店内服装的新图像——尚未充分利用这些强大的生成式解决方案的潜力。本研究提出LaDI-VTON，这是首个为虚拟试穿任务设计的潜在扩散文本反转增强模型。所提出的架构基于潜在扩散模型，并扩展了一个新颖的额外自编码器模块，该模块利用可学习的跳跃连接来增强生成过程，同时保留模型的特征。为了有效保持店内服装的纹理与细节，我们提出了一种文本反转组件，可以将服装的视觉特征映射到CLIP令牌嵌入空间，从而生成一组能够调节生成过程的伪词令牌嵌入。在Dress Code和VITON-HD数据集上的实验结果表明，我们的方法以显著优势超越了竞争对手，为这一任务树立了重要的里程碑。源代码和训练模型已在以下网址公开：https://github.com/miccunifi/ladi-vton。