The goal of image-based virtual try-on is to generate an image of the target person naturally wearing the given clothing. However, most existing methods solely focus on the frontal try-on using the frontal clothing. When the views of the clothing and person are significantly inconsistent, particularly when the person's view is non-frontal, the results are unsatisfactory. To address this challenge, we introduce Multi-View Virtual Try-ON (MV-VTON), which aims to reconstruct the dressing results of a person from multiple views using the given clothes. On the one hand, given that single-view clothes provide insufficient information for MV-VTON, we instead employ two images, i.e., the frontal and back views of the clothing, to encompass the complete view as much as possible. On the other hand, the diffusion models that have demonstrated superior abilities are adopted to perform our MV-VTON. In particular, we propose a view-adaptive selection method where hard-selection and soft-selection are applied to the global and local clothing feature extraction, respectively. This ensures that the clothing features are roughly fit to the person's view. Subsequently, we suggest a joint attention block to align and fuse clothing features with person features. Additionally, we collect a MV-VTON dataset, i.e., Multi-View Garment (MVG), in which each person has multiple photos with diverse views and poses. Experiments show that the proposed method not only achieves state-of-the-art results on MV-VTON task using our MVG dataset, but also has superiority on frontal-view virtual try-on task using VITON-HD and DressCode datasets. Codes and datasets will be publicly released at https://github.com/hywang2002/MV-VTON .

翻译：基于图像的虚拟试穿目标是生成目标人物自然穿着给定衣物的图像。然而，现有方法大多仅关注使用正面衣物进行正面试穿。当衣物与人物视角显著不一致，尤其是人物视角非正面时，其结果往往不尽如人意。为应对这一挑战，我们提出了多视角虚拟试穿（MV-VTON），旨在利用给定衣物从多视角重建人物的穿着效果。一方面，鉴于单视角衣物图像为MV-VTON提供的信息不足，我们转而采用两幅图像——即衣物的正面和背面视图——以尽可能覆盖完整视角。另一方面，我们采用已展现出卓越能力的扩散模型来执行MV-VTON。具体而言，我们提出了一种视角自适应选择方法，对全局和局部衣物特征提取分别应用硬选择与软选择，从而确保衣物特征与人物视角大致匹配。随后，我们引入联合注意力模块，以对齐和融合衣物特征与人物特征。此外，我们收集了MV-VTON数据集，即多视角服装数据集（MVG），其中每位人物均包含多张不同视角与姿态的照片。实验表明，所提方法不仅在使用我们的MVG数据集的MV-VTON任务上取得了最先进的结果，而且在基于VITON-HD和DressCode数据集的正面视角虚拟试穿任务上也展现出优越性。代码与数据集将公开发布于https://github.com/hywang2002/MV-VTON。