Recent advancements in Virtual Try-On (VTO) have demonstrated exceptional efficacy in generating realistic images and preserving garment details, largely attributed to the robust generative capabilities of text-to-image (T2I) diffusion backbones. However, the T2I models that underpin these methods have become outdated, thereby limiting the potential for further improvement in VTO. Additionally, current methods face notable challenges in accurately rendering text on garments without distortion and preserving fine-grained details, such as textures and material fidelity. The emergence of Diffusion Transformer (DiT) based T2I models has showcased impressive performance and offers a promising opportunity for advancing VTO. Directly applying existing VTO techniques to transformer-based T2I models is ineffective due to substantial architectural differences, which hinder their ability to fully leverage the models' advanced capabilities for improved text generation. To address these challenges and unlock the full potential of DiT-based T2I models for VTO, we propose TED-VITON, a novel framework that integrates a Garment Semantic (GS) Adapter for enhancing garment-specific features, a Text Preservation Loss to ensure accurate and distortion-free text rendering, and a constraint mechanism to generate prompts by optimizing Large Language Model (LLM). These innovations enable state-of-the-art (SOTA) performance in visual quality and text fidelity, establishing a new benchmark for VTO task. Project page: https://zhenchenwan.github.io/TED-VITON/

翻译：虚拟试穿技术的最新进展在生成逼真图像和保留服装细节方面展现出卓越成效，这很大程度上归功于文本到图像扩散主干模型强大的生成能力。然而，支撑这些方法的T2I模型已显陈旧，从而限制了VTO性能的进一步提升。此外，现有方法在准确且无失真地渲染服装上的文字，以及保留纹理和材质保真度等细粒度细节方面，仍面临显著挑战。基于扩散Transformer的T2I模型的出现展示了令人印象深刻的性能，为推进VTO技术提供了新的机遇。由于架构上的显著差异，直接将现有的VTO技术应用于基于Transformer的T2I模型效果不佳，这阻碍了其充分利用模型先进能力以改进文本生成。为解决这些挑战并充分释放基于DiT的T2I模型在VTO任务中的潜力，我们提出了TED-VITON，这是一个新颖的框架。它集成了一个用于增强服装特定特征的服装语义适配器，一个确保准确且无失真文本渲染的文本保留损失函数，以及一个通过优化大型语言模型来生成提示词的约束机制。这些创新实现了视觉质量和文本保真度的最先进性能，为VTO任务树立了新的基准。项目页面：https://zhenchenwan.github.io/TED-VITON/