Machine learning models always make a prediction, even when it is likely to be inaccurate. This behavior should be avoided in many decision support applications, where mistakes can have severe consequences. Albeit already studied in 1970, machine learning with rejection recently gained interest. This machine learning subfield enables machine learning models to abstain from making a prediction when likely to make a mistake. This survey aims to provide an overview on machine learning with rejection. We introduce the conditions leading to two types of rejection, ambiguity and novelty rejection, which we carefully formalize. Moreover, we review and categorize strategies to evaluate a model's predictive and rejective quality. Additionally, we define the existing architectures for models with rejection and describe the standard techniques for learning such models. Finally, we provide examples of relevant application domains and show how machine learning with rejection relates to other machine learning research areas.

翻译：机器学习模型即使在可能做出不准确预测时也会始终给出预测结果。在许多决策支持应用中，这一行为应予以避免，因为错误可能产生严重后果。尽管早在1970年就有相关研究，但带有拒绝选项的机器学习近来才引起广泛关注。这一机器学习子领域使模型能够在可能犯错时放弃预测。本综述旨在提供关于带有拒绝选项的机器学习的全面概述。我们引入导致两类拒绝（歧义拒绝与新奇拒绝）的条件，并对此进行严格形式化。此外，我们回顾并分类了评估模型预测质量与拒绝质量的策略。同时，我们定义了现有带有拒绝选项的模型架构，并描述了训练此类模型的标准技术。最后，我们提供了相关应用领域的示例，并展示了带有拒绝选项的机器学习与其他机器学习研究领域的关系。