Predictive models make mistakes. Hence, there is a need to quantify the uncertainty associated with their predictions. Conformal inference has emerged as a powerful tool to create statistically valid prediction regions around point predictions, but its naive application to regression problems yields non-adaptive regions. New conformal scores, often relying upon quantile regressors or conditional density estimators, aim to address this limitation. Although they are useful for creating prediction bands, these scores are detached from the original goal of quantifying the uncertainty around an arbitrary predictive model. This paper presents a new, model-agnostic family of methods to calibrate prediction intervals for regression problems with local coverage guarantees. Our approach is based on pursuing the coarsest partition of the feature space that approximates conditional coverage. We create this partition by training regression trees and Random Forests on conformity scores. Our proposal is versatile, as it applies to various conformity scores and prediction settings and demonstrates superior scalability and performance compared to established baselines in simulated and real-world datasets. We provide a Python package locart that implements our methods using the standard scikit-learn interface.
翻译:预测模型会产生错误,因此需要量化其预测结果的不确定性。共形推断已成为一种为点预测构建统计有效预测区间的强大工具,但其在回归问题中的直接应用会得到非自适应区间。新的共形分数通常依赖于分位数回归器或条件密度估计量,旨在解决这一局限性。尽管这些分数有助于构建预测带,但它们与量化任意预测模型不确定性的原始目标相脱离。本文提出了一族新型模型无关方法,用于校准具有局部覆盖保证的回归问题预测区间。我们的方法基于寻找特征空间的最粗略划分,以近似条件覆盖。我们通过基于共形分数训练回归树和随机森林来创建这种划分。该方案具有通用性,可应用于多种共形分数和预测场景,并在模拟和真实数据集上展现出相较于已有基线方法更优的可扩展性和性能。我们提供了Python包locart,通过标准的scikit-learn接口实现了我们的方法。