Features of the same sample generated by different pretrained models often exhibit inherently distinct feature distributions because of discrepancies in the model pretraining objectives or architectures. Learning invariant representations from large-scale unlabeled visual data with various pretrained models in a fully unsupervised transfer manner remains a significant challenge. In this paper, we propose a multiview self-representation learning (MSRL) method in which invariant representations are learned by exploiting the self-representation property of features across heterogeneous views. The features are derived from large-scale unlabeled visual data through transfer learning with various pretrained models and are referred to as heterogeneous multiview data. An individual linear model is stacked on top of its corresponding frozen pretrained backbone. We introduce an information-passing mechanism that relies on self-representation learning to support feature aggregation over the outputs of the linear model. Moreover, an assignment probability distribution consistency scheme is presented to guide multiview self-representation learning by exploiting complementary information across different views. Consequently, representation invariance across different linear models is enforced through this scheme. In addition, we provide a theoretical analysis of the information-passing mechanism, the assignment probability distribution consistency and the incremental views. Extensive experiments with multiple benchmark visual datasets demonstrate that the proposed MSRL method consistently outperforms several state-of-the-art approaches.

翻译：由于模型预训练目标或架构的差异，不同预训练模型生成的同一样本特征往往呈现出本质不同的特征分布。在完全无监督的迁移范式下，利用多种预训练模型从大规模无标注视觉数据中学习不变表征，仍然是一个重大挑战。本文提出一种多视图自表示学习方法，该方法通过利用异构视图间特征的自表示特性来学习不变表征。这些特征通过多种预训练模型的迁移学习从大规模无标注视觉数据中提取，被称为异构多视图数据。每个独立的线性模型堆叠在其对应的冻结预训练主干网络之上。我们引入一种基于自表示学习的信息传递机制，以支持对线性模型输出的特征聚合。此外，本文提出一种分配概率分布一致性方案，通过利用不同视图间的互补信息来指导多视图自表示学习。由此，该方案强制实现了不同线性模型间的表征不变性。另外，我们对信息传递机制、分配概率分布一致性及增量视图进行了理论分析。在多个基准视觉数据集上的大量实验表明，所提出的MSRL方法在多个指标上持续优于若干现有先进方法。