Self-training is a well-known approach for semi-supervised learning. It consists of iteratively assigning pseudo-labels to unlabeled data for which the model is confident and treating them as labeled examples. For neural networks, softmax prediction probabilities are often used as a confidence measure, although they are known to be overconfident, even for wrong predictions. This phenomenon is particularly intensified in the presence of sample selection bias, i.e., when data labeling is subject to some constraint. To address this issue, we propose a novel confidence measure, called $\mathcal{T}$-similarity, built upon the prediction diversity of an ensemble of linear classifiers. We provide the theoretical analysis of our approach by studying stationary points and describing the relationship between the diversity of the individual members and their performance. We empirically demonstrate the benefit of our confidence measure for three different pseudo-labeling policies on classification datasets of various data modalities. The code is available at https://github.com/ambroiseodt/tsim.

翻译：自训练是半监督学习中的一种经典方法，其核心思想是迭代地为模型高置信度的无标签数据分配伪标签，并将其视为有标签样本。对于神经网络而言，尽管softmax预测概率已知存在过度自信的问题（即使在错误预测时也如此），但仍常被用作置信度量。当存在样本选择偏差（即数据标记受限于某种约束条件）时，这一现象尤为加剧。为解决该问题，我们提出了一种新型置信度量——$\mathcal{T}$-相似度，该度量基于线性分类器集成的预测多样性构建。我们通过研究驻点并描述各成员分类器多样性与其性能之间的关系，对所提方法进行了理论分析。针对不同数据模态的分类数据集，我们采用三种伪标签策略进行了实验验证，结果表明我们的置信度量具有显著优势。相关代码已开源在https://github.com/ambroiseodt/tsim。