Unsupervised visible-infrared person re-identification (USL-VI-ReID) is a promising yet challenging retrieval task. The key challenges in USL-VI-ReID are to effectively generate pseudo-labels and establish pseudo-label correspondences across modalities without relying on any prior annotations. Recently, clustered pseudo-label methods have gained more attention in USL-VI-ReID. However, previous methods fell short of fully exploiting the individual nuances, as they simply utilized a single memory that represented an identity to establish cross-modality correspondences, resulting in ambiguous cross-modality correspondences. To address the problem, we propose a Multi-Memory Matching (MMM) framework for USL-VI-ReID. We first design a Cross-Modality Clustering (CMC) module to generate the pseudo-labels through clustering together both two modality samples. To associate cross-modality clustered pseudo-labels, we design a Multi-Memory Learning and Matching (MMLM) module, ensuring that optimization explicitly focuses on the nuances of individual perspectives and establishes reliable cross-modality correspondences. Finally, we design a Soft Cluster-level Alignment (SCA) module to narrow the modality gap while mitigating the effect of noise pseudo-labels through a soft many-to-many alignment strategy. Extensive experiments on the public SYSU-MM01 and RegDB datasets demonstrate the reliability of the established cross-modality correspondences and the effectiveness of our MMM. The source codes will be released.
翻译:无监督可见光-红外行人重识别(USL-VI-ReID)是一项前景广阔但极具挑战性的检索任务。USL-VI-ReID的关键挑战在于,在不依赖任何先验标注的情况下,有效生成伪标签并建立跨模态的伪标签对应关系。近年来,基于聚类的伪标签方法在USL-VI-ReID中受到更多关注。然而,先前的方法未能充分利用个体间的细微差异,因为它们仅使用代表一个身份的单记忆来建立跨模态对应关系,导致跨模态对应关系模糊不清。为解决此问题,我们提出了一种用于USL-VI-ReID的多记忆匹配(MMM)框架。我们首先设计了一个跨模态聚类(CMC)模块,通过对两种模态的样本进行联合聚类来生成伪标签。为了关联跨模态的聚类伪标签,我们设计了一个多记忆学习与匹配(MMLM)模块,确保优化过程明确关注个体视角的细微差异,并建立可靠的跨模态对应关系。最后,我们设计了一个软聚类级对齐(SCA)模块,通过一种软多对多对齐策略来缩小模态间隙,同时减轻噪声伪标签的影响。在公开的SYSU-MM01和RegDB数据集上进行的大量实验证明了所建立的跨模态对应关系的可靠性以及我们MMM方法的有效性。源代码将予以公开。