Given a limited labeling budget, active learning (AL) aims to sample the most informative instances from an unlabeled pool to acquire labels for subsequent model training. To achieve this, AL typically measures the informativeness of unlabeled instances based on uncertainty and diversity. However, it does not consider erroneous instances with their neighborhood error density, which have great potential to improve the model performance. To address this limitation, we propose $REAL$, a novel approach to select data instances with $\underline{R}$epresentative $\underline{E}$rrors for $\underline{A}$ctive $\underline{L}$earning. It identifies minority predictions as \emph{pseudo errors} within a cluster and allocates an adaptive sampling budget for the cluster based on estimated error density. Extensive experiments on five text classification datasets demonstrate that $REAL$ consistently outperforms all best-performing baselines regarding accuracy and F1-macro scores across a wide range of hyperparameter settings. Our analysis also shows that $REAL$ selects the most representative pseudo errors that match the distribution of ground-truth errors along the decision boundary. Our code is publicly available at https://github.com/withchencheng/ECML_PKDD_23_Real.

翻译：在有限的标注预算下，主动学习旨在从无标注数据池中采样最具信息量的实例，以获取标签用于后续模型训练。为此，主动学习通常基于不确定性和多样性衡量无标注实例的信息量。然而，该方法未考虑具有邻域错误密度的错误实例，而这些实例对提升模型性能具有巨大潜力。为解决这一局限，我们提出$REAL$——一种通过选择具有代表性的错误实例进行主动学习的新方法。该方法将聚类中的少数预测识别为“伪错误”，并基于估计的错误密度为每个聚类分配自适应采样预算。在五个文本分类数据集上的大量实验表明，$REAL$在广泛超参数设置下，其准确率和F1-macro分数始终优于所有最佳基线方法。我们的分析还显示，$REAL$能选择最具代表性的伪错误，这些伪错误在决策边界上匹配真实错误分布。我们的代码已公开于https://github.com/withchencheng/ECML_PKDD_23_Real。