In this paper, we consider the uncertainty quantification problem for regression models. Specifically, we consider an individual calibration objective for characterizing the quantiles of the prediction model. While such an objective is well-motivated from downstream tasks such as newsvendor cost, the existing methods have been largely heuristic and lack of statistical guarantee in terms of individual calibration. We show via simple examples that the existing methods focusing on population-level calibration guarantees such as average calibration or sharpness can lead to harmful and unexpected results. We propose simple nonparametric calibration methods that are agnostic of the underlying prediction model and enjoy both computational efficiency and statistical consistency. Our approach enables a better understanding of the possibility of individual calibration, and we establish matching upper and lower bounds for the calibration error of our proposed methods. Technically, our analysis combines the nonparametric analysis with a covering number argument for parametric analysis, which advances the existing theoretical analyses in the literature of nonparametric density estimation and quantile bandit problems. Importantly, the nonparametric perspective sheds new theoretical insights into regression calibration in terms of the curse of dimensionality and reconciles the existing results on the impossibility of individual calibration. To our knowledge, we make the first effort to reach both individual calibration and finite-sample guarantee with minimal assumptions in terms of conformal prediction. Numerical experiments show the advantage of such a simple approach under various metrics, and also under covariates shift. We hope our work provides a simple benchmark and a starting point of theoretical ground for future research on regression calibration.

翻译：本文针对回归模型的不确定性量化问题展开研究。具体而言，我们考虑了个体校准目标，用以刻画预测模型的分位数特性。尽管该目标在报童成本等下游任务中具有充分动机，但现有方法多为启发式，且在个体校准统计保证方面存在缺失。通过简单实例证明，现有聚焦于平均校准或锐度等群体级校准保证的方法可能引发有害且非预期的结果。我们提出了简单的非参数校准方法，这些方法对底层预测模型保持无关性，兼具计算效率与统计一致性。该方法有助于深入理解个体校准的可行性，并为所提方法的校准误差建立了匹配的上下界。在技术层面，我们的分析将非参数分析与覆盖数论证相结合以进行参数分析，推动了非参数密度估计与分位数老虎机问题领域现有理论分析的进步。重要的是，非参数视角为回归校准中的维度灾难问题提供了全新理论洞见，并调和了关于个体校准不可行性的现有结论。据我们所知，这是首次在保形预测框架下以最弱假设同时实现个体校准与有限样本保证。数值实验表明，该简单方法在多种评估指标及协变量偏移场景下均具有显著优势。期待本研究能为回归校准的未来研究提供简洁基准与理论基础起点。