The rapidly evolving fields of e-commerce and metaverse continue to seek innovative approaches to enhance the consumer experience. At the same time, recent advancements in the development of diffusion models have enabled generative networks to create remarkably realistic images. In this context, image-based virtual try-on, which consists in generating a novel image of a target model wearing a given in-shop garment, has yet to capitalize on the potential of these powerful generative solutions. This work introduces LaDI-VTON, the first Latent Diffusion textual Inversion-enhanced model for the Virtual Try-ON task. The proposed architecture relies on a latent diffusion model extended with a novel additional autoencoder module that exploits learnable skip connections to enhance the generation process preserving the model's characteristics. To effectively maintain the texture and details of the in-shop garment, we propose a textual inversion component that can map the visual features of the garment to the CLIP token embedding space and thus generate a set of pseudo-word token embeddings capable of conditioning the generation process. Experimental results on Dress Code and VITON-HD datasets demonstrate that our approach outperforms the competitors by a consistent margin, achieving a significant milestone for the task. Source code and trained models will be publicly released at: https://github.com/miccunifi/ladi-vton.

翻译：快速发展的电子商务和元宇宙领域持续探索创新方法以提升消费者体验。与此同时，扩散模型的最新进展使生成网络能够创造极其逼真的图像。在此背景下，基于图像的虚拟试穿（生成目标模特穿着指定店内服装的新图像）尚未充分利用这些强大生成式解决方案的潜力。本研究提出LaDI-VTON，这是首个面向虚拟试穿任务的潜在扩散文本反转增强模型。所提出的架构基于潜在扩散模型，并扩展了一个新型额外自编码器模块，该模块利用可学习跳跃连接以在保留模特特征的同时增强生成过程。为有效保持店内服装的纹理与细节，我们提出文本反转组件，能将服装视觉特征映射至CLIP词元嵌入空间，从而生成一组伪词元嵌入以调控生成过程。在Dress Code与VITON-HD数据集上的实验结果表明，本方法以显著优势超越现有竞争方法，为该任务树立了重要里程碑。源代码与预训练模型将在https://github.com/miccunifi/ladi-vton 公开发布。