Conformal inference is a popular tool for constructing prediction intervals (PI). We consider here the scenario of post-selection/selective conformal inference, that is PIs are reported only for individuals selected from an unlabeled test data. To account for multiplicity, we develop a general split conformal framework to construct selective PIs with the false coverage-statement rate (FCR) control. We first investigate the Benjamini and Yekutieli (2005)'s FCR-adjusted method in the present setting, and show that it is able to achieve FCR control but yields uniformly inflated PIs. We then propose a novel solution to the problem, named as Selective COnditional conformal Predictions (SCOP), which entails performing selection procedures on both calibration set and test set and construct marginal conformal PIs on the selected sets by the aid of conditional empirical distribution obtained by the calibration set. Under a unified framework and exchangeable assumptions, we show that the SCOP can exactly control the FCR. More importantly, we provide non-asymptotic miscoverage bounds for a general class of selection procedures beyond exchangeablity and discuss the conditions under which the SCOP is able to control the FCR. As special cases, the SCOP with quantile-based selection or conformal p-values-based multiple testing procedures enjoys valid coverage guarantee under mild conditions. Numerical results confirm the effectiveness and robustness of SCOP in FCR control and show that it achieves more narrowed PIs over existing methods in many settings.
翻译:共形推断是构建预测区间(PI)的常用工具。本文考虑后选择/选择性共形推断场景,即仅针对从无标签测试数据中选定的个体报告预测区间。为应对多重性问题,我们开发了一个通用的分割共形框架,用于构建具有误覆率(FCR)控制的选择性预测区间。首先,我们研究了Benjamini与Yekutieli(2005)的FCR校正方法在本场景中的应用,证明其虽能实现FCR控制,但会导致预测区间整体膨胀。随后我们提出一种新颖解决方案——选择性条件共形预测(SCOP),该方法需同时在校准集与测试集上执行选择程序,并借助校准集获取的条件经验分布为选定集合构建边缘共形预测区间。在统一框架与可交换性假设下,我们证明SCOP能精确控制FCR。更重要的是,我们为突破可交换性假设的一般选择程序类提供了非渐近的误覆盖界,并讨论了SCOP实现FCR控制的充分条件。作为特例,基于分位数选择或共形p值多重检验程序的SCOP在温和条件下享有有效的覆盖保证。数值结果验证了SCOP在FCR控制中的有效性与鲁棒性,并表明其在多种场景下能生成比现有方法更窄的预测区间。