Given a limited labeling budget, active learning (AL) aims to sample the most informative instances from an unlabeled pool to acquire labels for subsequent model training. To achieve this, AL typically measures the informativeness of unlabeled instances based on uncertainty and diversity. However, it does not consider erroneous instances with their neighborhood error density, which have great potential to improve the model performance. To address this limitation, we propose $REAL$, a novel approach to select data instances with $\underline{R}$epresentative $\underline{E}$rrors for $\underline{A}$ctive $\underline{L}$earning. It identifies minority predictions as \emph{pseudo errors} within a cluster and allocates an adaptive sampling budget for the cluster based on estimated error density. Extensive experiments on five text classification datasets demonstrate that $REAL$ consistently outperforms all best-performing baselines regarding accuracy and F1-macro scores across a wide range of hyperparameter settings. Our analysis also shows that $REAL$ selects the most representative pseudo errors that match the distribution of ground-truth errors along the decision boundary. Our code is publicly available at https://github.com/withchencheng/ECML_PKDD_23_Real.

翻译：在标注预算有限的情况下，主动学习（AL）旨在从未标注池中采样最具信息量的实例以获取标签，用于后续模型训练。为此，AL通常基于不确定性和多样性来评估未标注实例的信息量。然而，它并未考虑与其邻域错误密度相关的错误实例，而这些实例具有显著提升模型性能的潜力。为解决这一局限，我们提出$REAL$，一种通过选择具有代表性错误（Rep$\underline{r}$esentative $\underline{E}$rrors）的实例进行主动学习（$\underline{A}$ctive $\underline{L}$earning）的新方法。该方法将聚类内的少数预测识别为伪错误（pseudo errors），并基于估计的错误密度为聚类分配自适应采样预算。在五个文本分类数据集上的大量实验表明，$REAL$在广泛超参数设置下，在准确率和F1-宏平均分数上始终优于所有最佳基线方法。我们的分析还显示，$REAL$选择的代表性伪错误能够匹配决策边界上真实错误的分布。我们的代码已公开于https://github.com/withchencheng/ECML_PKDD_23_Real。