Learning with abstention is a key scenario where the learner can abstain from making a prediction at some cost. In this paper, we analyze the score-based formulation of learning with abstention in the multi-class classification setting. We introduce new families of surrogate losses for the abstention loss function, which include the state-of-the-art surrogate losses in the single-stage setting and a novel family of loss functions in the two-stage setting. We prove strong non-asymptotic and hypothesis set-specific consistency guarantees for these surrogate losses, which upper-bound the estimation error of the abstention loss function in terms of the estimation error of the surrogate loss. Our bounds can help compare different score-based surrogates and guide the design of novel abstention algorithms by minimizing the proposed surrogate losses. We experimentally evaluate our new algorithms on CIFAR-10, CIFAR-100, and SVHN datasets and the practical significance of our new surrogate losses and two-stage abstention algorithms. Our results also show that the relative performance of the state-of-the-art score-based surrogate losses can vary across datasets.

翻译：弃权学习是一个关键场景，学习者可以以一定成本放弃做出预测。本文分析了多类分类场景中基于分数的弃权学习形式。我们为弃权损失函数引入了新的替代损失函数族，包括单阶段设置中最先进的替代损失函数以及两阶段设置中新颖的损失函数族。我们证明了这些替代损失函数具有强非渐近性和假设集特定的一致性保证，这些保证通过替代损失函数的估计误差对弃权损失函数的估计误差进行上限约束。我们的界限有助于比较不同的基于分数的替代函数，并通过最小化所提出的替代损失函数来指导新型弃权算法的设计。我们在CIFAR-10、CIFAR-100和SVHN数据集上实验评估了新算法，并验证了新替代损失函数和两阶段弃权算法的实际意义。结果还表明，最先进的基于分数的替代损失函数的相对性能可能因数据集而异。