While the traditional viewpoint in machine learning and statistics assumes training and testing samples come from the same population, practice belies this fiction. One strategy -- coming from robust statistics and optimization -- is thus to build a model robust to distributional perturbations. In this paper, we take a different approach to describe procedures for robust predictive inference, where a model provides uncertainty estimates on its predictions rather than point predictions. We present a method that produces prediction sets (almost exactly) giving the right coverage level for any test distribution in an $f$-divergence ball around the training population. The method, based on conformal inference, achieves (nearly) valid coverage in finite samples, under only the condition that the training data be exchangeable. An essential component of our methodology is to estimate the amount of expected future data shift and build robustness to it; we develop estimators and prove their consistency for protection and validity of uncertainty estimates under shifts. By experimenting on several large-scale benchmark datasets, including Recht et al.'s CIFAR-v4 and ImageNet-V2 datasets, we provide complementary empirical results that highlight the importance of robust predictive validity.

翻译：虽然机器学习和统计学中的传统观点假设训练样本与测试样本来自同一总体，但实践往往与这一假设相悖。源自鲁棒统计学与优化领域的一种策略是构建对分布扰动具有鲁棒性的模型。本文则采用不同方法，旨在描述鲁棒预测推断的流程——模型为其预测提供不确定性估计而非点估计。我们提出一种方法，能够生成预测集合，对于训练总体周围$f$散度球内的任意测试分布，（几乎完全）提供正确的覆盖水平。该方法基于共形推断，仅需训练数据满足可交换性条件，即可在有限样本下实现（近乎）有效的覆盖。本方法的核心环节是预估预期未来数据偏移量并构建针对该偏移的鲁棒性；我们开发了相应的估计量，并证明了其在偏移条件下对不确定性估计的保护性与有效性具有一致性。通过在多个大规模基准数据集（包括Recht等人提出的CIFAR-v4和ImageNet-V2数据集）上的实验，我们提供了补充性实证结果，凸显了鲁棒预测有效性的重要意义。