Transferability estimation aims to provide heuristics for quantifying how suitable a pre-trained model is for a specific downstream task, without fine-tuning them all. Prior studies have revealed that well-trained models exhibit the phenomenon of Neural Collapse. Based on a widely used neural collapse metric in existing literature, we observe a strong correlation between the neural collapse of pre-trained models and their corresponding fine-tuned models. Inspired by this observation, we propose a novel method termed Fair Collapse (FaCe) for transferability estimation by comprehensively measuring the degree of neural collapse in the pre-trained model. Typically, FaCe comprises two different terms: the variance collapse term, which assesses the class separation and within-class compactness, and the class fairness term, which quantifies the fairness of the pre-trained model towards each class. We investigate FaCe on a variety of pre-trained classification models across different network architectures, source datasets, and training loss functions. Results show that FaCe yields state-of-the-art performance on different tasks including image classification, semantic segmentation, and text classification, which demonstrate the effectiveness and generalization of our method.

翻译：迁移性估计旨在提供启发式方法，用于量化预训练模型对特定下游任务的适用性，而无需对所有模型进行微调。先前研究表明，训练良好的模型会表现出神经坍缩现象。基于现有文献中广泛使用的神经坍缩度量，我们观察到预训练模型的神经坍缩与其对应微调模型之间存在强相关性。受此启发，我们提出一种名为公平坍缩（FaCe）的新方法，通过全面测量预训练模型中的神经坍缩程度来进行迁移性估计。通常，FaCe包含两个不同的项：方差坍缩项，用于评估类别分离和类内紧凑性；以及类别公平性项，用于量化预训练模型对每个类别的公平性。我们在不同网络架构、源数据集和训练损失函数的多种预训练分类模型上研究了FaCe。结果表明，FaCe在图像分类、语义分割和文本分类等不同任务上均取得了最先进的性能，证明了我们方法的有效性和泛化性。