Credal predictors are models that are aware of epistemic uncertainty and produce a convex set of probabilistic predictions. They offer a principled way to quantify predictive epistemic uncertainty (EU) and have been shown to improve model robustness in various settings. However, most state-of-the-art methods mainly define EU as disagreement caused by random training initializations, which mostly reflects sensitivity to optimization randomness rather than uncertainty from deeper sources. To address this, we define EU as disagreement among models trained with varying relaxations of the i.i.d. assumption between training and test data. Based on this idea, we propose CreDRO, which learns an ensemble of plausible models through distributionally robust optimization. As a result, CreDRO captures EU not only from training randomness but also from meaningful disagreement due to potential distribution shifts between training and test data. Empirical results show that CreDRO consistently outperforms existing credal methods on tasks such as out-of-distribution detection across multiple benchmarks and selective classification in medical applications.
翻译:可信预测器是能够感知认知不确定性并生成凸概率预测集的模型。这类模型为量化预测性认知不确定性提供了理论框架,并已被证明能在多种场景下提升模型的鲁棒性。然而,当前主流方法通常将认知不确定性定义为由随机训练初始化引起的预测分歧,这主要反映的是对优化随机性的敏感度,而非更深层次的不确定性来源。为解决此问题,我们将认知不确定性定义为:在训练数据与测试数据间独立同分布假设的不同松弛条件下,所训练模型产生的预测分歧。基于这一理念,我们提出CreDRO方法,通过分布鲁棒优化学习一组合理模型的集成。因此,CreDRO不仅能够捕捉来自训练随机性的认知不确定性,还能捕获因训练与测试数据间潜在分布偏移所导致的有意义分歧。实验结果表明,在多项基准测试的分布外检测任务以及医疗应用中的选择性分类任务中,CreDRO均持续优于现有的可信预测方法。