We introduce and study the problem of calibrating conditional risk, which involves estimating the expected loss of a prediction model conditional on input features. We analyze this problem in both classification and regression settings and show that it is fundamentally equivalent to a standard regression task. For classification settings, we further establish a connection between conditional risk calibration and individual/conditional probability calibration, and develop theoretical insights for the performance metric. This reveals that while conditional risk calibration is related to existing uncertainty quantification problems, it remains a distinct and standalone machine learning problem. Empirically, we validate our theoretical findings and demonstrate the practical implications of conditional risk calibration in the learning to defer (L2D) framework. Our systematic experiments provide both qualitative and quantitative assessments, offering guidance for future research in uncertainty-aware decision-making.

翻译：我们引入并研究了校准条件风险的问题，该问题涉及估计预测模型在输入特征条件下的期望损失。我们在分类和回归场景中分析了该问题，并表明其本质上等同于标准回归任务。对于分类场景，我们进一步建立了条件风险校准与个体/条件概率校准之间的联系，并为性能指标提供了理论洞见。这表明，虽然条件风险校准与现有不确定性量化问题相关，但它仍是一个独立且独特的机器学习问题。在实验方面，我们验证了理论发现，并展示了条件风险校准在学习延迟决策（L2D）框架中的实际意义。系统的实验提供了定性和定量评估，为未来不确定性感知决策研究提供了指导。