In deploying artificial intelligence (AI) models, selective prediction offers the option to abstain from making a prediction when uncertain about model quality. To fulfill its promise, it is crucial to enforce strict and precise error control over cases where the model is trusted. We propose Selective Conformal Risk control with E-values (SCoRE), a new framework for deriving such decisions for any trained model and any user-defined, bounded and continuously-valued risk. SCoRE offers two types of guarantees on the risk among ``positive'' cases in which the system opts to trust the model. Built upon conformal inference and hypothesis testing ideas, SCoRE first constructs a class of (generalized) e-values, which are non-negative random variables whose product with the unknown risk has expectation no greater than one. Such a property is ensured by data exchangeability without requiring any modeling assumptions. Passing these e-values on to hypothesis testing procedures, we yield the binary trust decisions with finite-sample error control. SCoRE avoids the need of uniform concentration, and can be readily extended to settings with distribution shifts. We evaluate the proposed methods with simulations and demonstrate their efficacy through applications to error management in drug discovery, health risk prediction, and large language models.

翻译：在部署人工智能模型时，选择性预测提供了当对模型质量不确定时放弃预测的选项。为实现其预期效果，必须对模型被信任的情况实施严格且精确的误差控制。我们提出了基于E值的选择性共形风险控制框架（SCoRE），这是一个为任意训练模型及用户定义的、有界且连续取值的风险提供此类决策的新框架。SCoRE在系统选择信任模型的"正例"中提供两种风险保证。该框架基于共形推断和假设检验思想，首先构建一类（广义）E值——其为非负随机变量，其与未知风险的乘积期望不超过1。该性质由数据可交换性保证，无需任何建模假设。将这些E值传递给假设检验程序后，我们可获得具有有限样本误差控制的二元信任决策。SCoRE避免了均匀集中性的需求，并可轻松扩展至分布偏移场景。我们通过模拟实验评估了所提方法，并在药物发现错误管理、健康风险预测及大语言模型中的应用验证了其有效性。