Learning with rejection is a prototypical model for studying the interaction between humans and AI on prediction tasks. The model has two components, a predictor and a rejector. Upon the arrival of a sample, the rejector first decides whether to accept it; if accepted, the predictor fulfills the prediction task, and if rejected, the prediction will be deferred to humans. The learning problem requires learning a predictor and a rejector simultaneously. This changes the structure of the conventional loss function and often results in non-convexity and inconsistency issues. For the classification with rejection problem, several works develop surrogate losses for the jointly learning with provable consistency guarantees; in parallel, there has been less work for the regression counterpart. We study the regression with rejection (RwR) problem and investigate the no-rejection learning strategy which treats the RwR problem as a standard regression task to learn the predictor. We establish that the suboptimality of the no-rejection learning strategy observed in the literature can be mitigated by enlarging the function class of the predictor. Then we introduce the truncated loss to single out the learning for the predictor and we show that a consistent surrogate property can be established for the predictor individually in an easier way than for the predictor and the rejector jointly. Our findings advocate for a two-step learning procedure that first uses all the data to learn the predictor and then calibrates the prediction loss for the rejector. It is better aligned with the common intuition that more data samples will lead to a better predictor and it calls for more efforts on a better design of calibration algorithms for learning the rejector. While our discussions mainly focus on the regression problem, the theoretical results and insights generalize to the classification problem as well.

翻译：带拒绝学习是研究人类与人工智能在预测任务中交互的原型模型。该模型包含两个组件：预测器和拒绝器。当样本到达时，拒绝器首先决定是否接受该样本；若接受，则由预测器完成预测任务；若拒绝，则预测任务将转交给人类处理。该学习问题要求同时学习预测器和拒绝器，这一特性改变了传统损失函数的结构，常导致非凸性和不一致性问题。针对带拒绝分类问题，已有若干研究提出了具有可证明一致性的联合学习替代损失；然而，其回归对应问题的研究相对较少。本文研究带拒绝回归问题，并探讨将带拒绝回归视为标准回归任务来学习预测器的无拒绝学习策略。我们证明，通过扩宽预测器的函数类，可以缓解文献中观察到的无拒绝学习策略的次优性。随后，我们引入截断损失单独分离预测器的学习过程，并证明相较于对预测器和拒绝器的联合学习，可以更简便地为预测器单独建立一致性替代性质。我们的研究结果支持采用两步学习流程：首先使用全部数据学习预测器，然后为拒绝器校正预测损失。这一方法更好地契合了“更多数据样本将带来更优预测器”的普遍直觉，并呼吁在拒绝器学习的校正算法设计方面投入更多研究。虽然本文主要聚焦回归问题，但理论结果和洞见同样可推广至分类问题。