Should prediction models always deliver a prediction? In the pursuit of maximum predictive performance, critical considerations of reliability and fairness are often overshadowed, particularly when it comes to the role of uncertainty. Selective regression, also known as the "reject option," allows models to abstain from predictions in cases of considerable uncertainty. Initially proposed seven decades ago, approaches to selective regression have mostly focused on distribution-based proxies for measuring uncertainty, particularly conditional variance. However, this focus neglects the significant influence of model-specific biases on a model's performance. In this paper, we propose a novel approach to selective regression by leveraging conformal prediction, which provides grounded confidence measures for individual predictions based on model-specific biases. In addition, we propose a standardized evaluation framework to allow proper comparison of selective regression approaches. Via an extensive experimental approach, we demonstrate how our proposed approach, conformalized selective regression, demonstrates an advantage over multiple state-of-the-art baselines.

翻译：预测模型是否总是需要做出预测？在追求最大预测性能的过程中，对可靠性和公平性的关键考量往往被忽视，尤其是在不确定性作用方面。选择性回归（也称为“拒绝选项”）允许模型在存在显著不确定性时放弃预测。这一概念于七十年前首次提出，但相关方法主要集中在基于分布的不确定性度量代理上，尤其是条件方差。然而，这种关注忽略了模型特定偏差对性能的显著影响。本文提出了一种利用符合预测（conformal prediction）的新型选择性回归方法，该方法基于模型特定偏差为单个预测提供有依据的置信度度量。此外，我们提出了一种标准化评估框架，以便对选择性回归方法进行合理比较。通过广泛的实验方法，我们证明了所提出的方法——符合化选择性回归——相较于多个最先进的基准方法具有优势。