Selective classification (or classification with a reject option) pairs a classifier with a selection function to determine whether or not a prediction should be accepted. This framework trades off coverage (probability of accepting a prediction) with predictive performance, typically measured by distributive loss functions. In many application scenarios, such as credit scoring, performance is instead measured by ranking metrics, such as the Area Under the ROC Curve (AUC). We propose a model-agnostic approach to associate a selection function to a given probabilistic binary classifier. The approach is specifically targeted at optimizing the AUC. We provide both theoretical justifications and a novel algorithm, called AUCROSS, to achieve such a goal. Experiments show that our method succeeds in trading-off coverage for AUC, improving over existing selective classification methods targeted at optimizing accuracy.

翻译：选择性分类（或称带拒绝选项的分类）将分类器与选择函数结合，用于决定是否接受某个预测。该框架在覆盖率（接受预测的概率）与预测性能之间进行权衡，后者通常通过分布损失函数来衡量。在信用评分等许多应用场景中，性能转而通过排序指标（如ROC曲线下面积，AUC）来衡量。我们提出了一种模型无关的方法，为给定的概率二分类器关联一个选择函数。该方法专门针对优化AUC而设计。我们提供了理论论证，并提出了一种名为AUCROSS的新算法来实现这一目标。实验表明，我们的方法成功地在覆盖率与AUC之间实现了权衡，并且相较于现有的以优化准确率为目标的选择性分类方法，性能有所提升。