For some classification scenarios, it is desirable to use only those classification instances that a trained model associates with a high certainty. To obtain such high-certainty instances, previous work has proposed accuracy-reject curves. Reject curves allow to evaluate and compare the performance of different certainty measures over a range of thresholds for accepting or rejecting classifications. However, the accuracy may not be the most suited evaluation metric for all applications, and instead precision or recall may be preferable. This is the case, for example, for data with imbalanced class distributions. We therefore propose reject curves that evaluate precision and recall, the recall-reject curve and the precision-reject curve. Using prototype-based classifiers from learning vector quantization, we first validate the proposed curves on artificial benchmark data against the accuracy reject curve as a baseline. We then show on imbalanced benchmarks and medical, real-world data that for these scenarios, the proposed precision- and recall-curves yield more accurate insights into classifier performance than accuracy reject curves.

翻译：在某些分类场景中，我们希望仅使用训练模型具有高置信度的分类实例。为获取此类高置信度实例，此前研究提出了准确率-拒绝曲线。拒绝曲线允许在分类接受/拒绝阈值范围内评估和比较不同置信度测量的性能。然而，准确率并非适用于所有应用的最佳评估指标，在某些情况下精确率或召回率更为合适，例如在类别分布不平衡的数据中。为此，我们提出评估精确率和召回率的拒绝曲线——召回率-拒绝曲线和精确率-拒绝曲线。基于学习向量量化的原型分类器，我们首先在人工基准数据上以准确率拒绝曲线为基线验证所提曲线。随后在不平衡基准数据和真实世界医疗数据上证明，相较于准确率拒绝曲线，本文提出的精确率与召回率曲线能为这些场景下的分类器性能提供更准确的洞察。