While the traditional viewpoint in machine learning and statistics assumes training and testing samples come from the same population, practice belies this fiction. One strategy -- coming from robust statistics and optimization -- is thus to build a model robust to distributional perturbations. In this paper, we take a different approach to describe procedures for robust predictive inference, where a model provides uncertainty estimates on its predictions rather than point predictions. We present a method that produces prediction sets (almost exactly) giving the right coverage level for any test distribution in an $f$-divergence ball around the training population. The method, based on conformal inference, achieves (nearly) valid coverage in finite samples, under only the condition that the training data be exchangeable. An essential component of our methodology is to estimate the amount of expected future data shift and build robustness to it; we develop estimators and prove their consistency for protection and validity of uncertainty estimates under shifts. By experimenting on several large-scale benchmark datasets, including Recht et al.'s CIFAR-v4 and ImageNet-V2 datasets, we provide complementary empirical results that highlight the importance of robust predictive validity.

翻译：尽管机器学习和统计学中的传统观点假设训练样本与测试样本来自同一总体，但实际应用往往背离这一假设。一种策略——源自鲁棒统计与优化——是构建对分布扰动具有鲁棒性的模型。本文另辟蹊径，描述了用于鲁棒预测推断的流程方法，其中模型提供的是对其预测的不确定性估计而非点预测。我们提出了一种方法，能够（近乎精确地）生成预测集，使得在训练总体周围$f$-散度球内的任何测试分布上，预测集都能给出正确的覆盖水平。该方法基于共形推断，在仅有训练数据可交换性这一条件下，即可在有限样本中实现（近乎）有效的覆盖。我们方法论的核心是估计预期的未来数据偏移量，并针对此偏移构建鲁棒性；我们开发了估计量，并证明了其在偏移下对不确定性估计的保护性与有效性的相合性。通过对多个大规模基准数据集（包括Recht等人的CIFAR-v4和ImageNet-V2数据集）进行实验，我们提供了补充性经验结果，凸显了鲁棒预测有效性的重要性。