In decision-making guided by machine learning, decision-makers often take identical actions in contexts with identical predicted outcomes. Conformal prediction helps decision-makers quantify outcome uncertainty for actions, allowing for better risk management. Inspired by this perspective, we introduce self-consistent conformal prediction, which yields both Venn-Abers calibrated predictions and conformal prediction intervals that are valid conditional on actions prompted by model predictions. Our procedure can be applied post-hoc to any black-box predictor to provide rigorous, action-specific decision-making guarantees. Numerical experiments show our approach strikes a balance between interval efficiency and conditional validity.
翻译:在机器学习指导的决策过程中,决策者通常在预测结果相同的上下文中采取相同行动。共形预测帮助决策者量化行动结果的不确定性,从而实现更优的风险管理。受该视角启发,我们提出自洽共形预测方法,该方法既能产生基于Venn-Abers校准的预测结果,又能构建在模型预测所触发的行动条件下保持有效性的共形预测区间。本方法可事后应用于任何黑箱预测器,为行动导向的决策提供严格的保障。数值实验表明,该方法在区间效率与条件有效性之间取得了良好平衡。