In this paper, we consider the uncertainty quantification problem for regression models. Specifically, we consider an individual calibration objective for characterizing the quantiles of the prediction model. While such an objective is well-motivated from downstream tasks such as newsvendor cost, the existing methods have been largely heuristic and lack of statistical guarantee in terms of individual calibration. We show via simple examples that the existing methods focusing on population-level calibration guarantees such as average calibration or sharpness can lead to harmful and unexpected results. We propose simple nonparametric calibration methods that are agnostic of the underlying prediction model and enjoy both computational efficiency and statistical consistency. Our approach enables a better understanding of the possibility of individual calibration, and we establish matching upper and lower bounds for the calibration error of our proposed methods. Technically, our analysis combines the nonparametric analysis with a covering number argument for parametric analysis, which advances the existing theoretical analyses in the literature of nonparametric density estimation and quantile bandit problems. Importantly, the nonparametric perspective sheds new theoretical insights into regression calibration in terms of the curse of dimensionality and reconciles the existing results on the impossibility of individual calibration. Numerical experiments show the advantage of such a simple approach under various metrics, and also under covariates shift. We hope our work provides a simple benchmark and a starting point of theoretical ground for future research on regression calibration.

翻译：本文研究了回归模型的不确定性量化问题。具体而言，我们考虑了一个针对预测模型分位数表征的个体校准目标。尽管该目标因下游任务（如报童成本）而具有充分动机，但现有方法大多基于启发式策略，且在个体校准方面缺乏统计保证。通过简单实例，我们证明现有聚焦于群体级校准保证（如平均校准或锐度）的方法可能导致有害且意外的结果。我们提出了简单的非参数校准方法，这些方法与底层预测模型无关，兼具计算效率与统计一致性。该方法使我们能更深入地理解个体校准的可能性，并为所提方法建立了校准误差的匹配上下界。在理论上，我们的分析将非参数分析与参数分析的覆盖数论证相结合，推进了非参数密度估计与分位数老虎机文献中的现有理论分析。重要的是，非参数视角在维度灾难方面为回归校准提供了新的理论见解，并调和了现有关于个体校准不可能性的结论。数值实验表明，这种简单方法在多种度量标准及协变量偏移下均具优势。我们希望这项工作将为回归校准的未来研究提供一个简单基准与理论出发点。