The goal of image-based virtual try-on is to generate an image of the target person naturally wearing the given clothing. However, most existing methods solely focus on the frontal try-on using the frontal clothing. When the views of the clothing and person are significantly inconsistent, particularly when the person's view is non-frontal, the results are unsatisfactory. To address this challenge, we introduce Multi-View Virtual Try-ON (MV-VTON), which aims to reconstruct the dressing results of a person from multiple views using the given clothes. On the one hand, given that single-view clothes provide insufficient information for MV-VTON, we instead employ two images, i.e., the frontal and back views of the clothing, to encompass the complete view as much as possible. On the other hand, the diffusion models that have demonstrated superior abilities are adopted to perform our MV-VTON. In particular, we propose a view-adaptive selection method where hard-selection and soft-selection are applied to the global and local clothing feature extraction, respectively. This ensures that the clothing features are roughly fit to the person's view. Subsequently, we suggest a joint attention block to align and fuse clothing features with person features. Additionally, we collect a MV-VTON dataset, i.e., Multi-View Garment (MVG), in which each person has multiple photos with diverse views and poses. Experiments show that the proposed method not only achieves state-of-the-art results on MV-VTON task using our MVG dataset, but also has superiority on frontal-view virtual try-on task using VITON-HD and DressCode datasets. Codes and datasets will be publicly released at https://github.com/hywang2002/MV-VTON .

翻译：基于图像的虚拟试穿旨在生成目标人物自然穿着给定衣物的图像。然而，现有方法大多仅关注使用正面衣物进行正面试穿。当衣物与人物的视角存在显著不一致时，尤其当人物视角非正面时，结果往往不尽如人意。为应对这一挑战，我们提出多视角虚拟试穿（MV-VTON），旨在利用给定衣物从多个视角重建人物的穿着效果。一方面，考虑到单视角衣物为MV-VTON提供的信息不足，我们转而采用两幅图像（即衣物的正面和背面视角）以尽可能覆盖完整视角。另一方面，我们采用已展现出卓越能力的扩散模型来实现MV-VTON。具体而言，我们提出一种视角自适应选择方法，对全局和局部衣物特征提取分别应用硬选择与软选择，从而确保衣物特征大致适配人物视角。随后，我们设计了一个联合注意力模块来对齐并融合衣物特征与人物特征。此外，我们构建了MV-VTON数据集——多视角服装（MVG）数据集，其中每位人物均拥有多张不同视角与姿态的照片。实验表明，所提方法不仅在使用MVG数据集的MV-VTON任务上取得了最先进结果，在使用VITON-HD和DressCode数据集的正面视角虚拟试穿任务中也展现出优越性。代码与数据集将在https://github.com/hywang2002/MV-VTON 公开。