Conformal Prediction (CP) algorithms estimate the uncertainty of a prediction model by calibrating its outputs on labeled data. The same calibration scheme usually applies to any model and data without modifications. The obtained prediction intervals are valid by construction but could be inefficient, i.e. unnecessarily big, if the prediction errors are not uniformly distributed over the input space. We present a general scheme to localize the intervals by training the calibration process. The standard prediction error is replaced by an optimized distance metric that depends explicitly on the object attributes. Learning the optimal metric is equivalent to training a Normalizing Flow that acts on the joint distribution of the errors and the inputs. Unlike the Error Reweighting CP algorithm of Papadopoulos et al. (2008), the framework allows estimating the gap between nominal and empirical conditional validity. The approach is compatible with existing locally-adaptive CP strategies based on re-weighting the calibration samples and applies to any point-prediction model without retraining.
翻译:保形预测(CP)算法通过在标注数据上校准预测模型的输出,来估计其预测的不确定性。相同的校准方案通常适用于任何模型和数据,无需修改。所获得的预测区间在构造上是有效的,但如果预测误差在输入空间上不是均匀分布的,则可能是低效的,即不必要地过大。我们提出了一种通过训练校准过程来局部化区间的一般方案。标准预测误差被替换为一种明确依赖于对象属性的优化距离度量。学习最优度量等价于训练一个作用于误差与输入联合分布的归一化流。与Papadopoulos等人(2008)的误差重加权CP算法不同,该框架允许估计名义条件有效性与经验条件有效性之间的差距。该方法与基于重新加权校准样本的现有局部自适应CP策略兼容,并且适用于任何点预测模型而无需重新训练。