Conformal inference is a popular tool for constructing prediction intervals (PI). We consider here the scenario of post-selection/selective conformal inference, that is PIs are reported only for individuals selected from an unlabeled test data. To account for multiplicity, we develop a general split conformal framework to construct selective PIs with the false coverage-statement rate (FCR) control. We first investigate the Benjamini and Yekutieli (2005)'s FCR-adjusted method in the present setting, and show that it is able to achieve FCR control but yields uniformly inflated PIs. We then propose a novel solution to the problem, named as Selective COnditional conformal Predictions (SCOP), which entails performing selection procedures on both calibration set and test set and construct marginal conformal PIs on the selected sets by the aid of conditional empirical distribution obtained by the calibration set. Under a unified framework and exchangeable assumptions, we show that the SCOP can exactly control the FCR. More importantly, we provide non-asymptotic miscoverage bounds for a general class of selection procedures beyond exchangeablity and discuss the conditions under which the SCOP is able to control the FCR. As special cases, the SCOP with quantile-based selection or conformal p-values-based multiple testing procedures enjoys valid coverage guarantee under mild conditions. Numerical results confirm the effectiveness and robustness of SCOP in FCR control and show that it achieves more narrowed PIs over existing methods in many settings.
翻译:共形推断是一种构建预测区间(PI)的流行工具。本文研究后选择/选择性共形推断场景,即仅对从未标注测试数据中选出的个体报告预测区间。为处理多重性问题,我们开发了一个通用的分割共形框架,用于构建具有错误覆盖陈述率(FCR)控制的选择性预测区间。我们首先研究了Benjamini与Yekutieli(2005)的FCR校正方法在当前场景下的表现,发现该方法虽能实现FCR控制,但会导致预测区间整体过宽。随后我们提出了一种名为"选择性条件共形预测"(SCOP)的新解决方案,其核心思想是在校准集和测试集上分别执行选择程序,并借助校准集获得的条件经验分布,为所选集合构建边界共形预测区间。在统一框架与可交换性假设下,我们证明了SCOP能精确控制FCR。更重要的是,我们为非可交换性条件下的广义选择程序类提供了非渐近误覆盖边界,并讨论了SCOP实现FCR控制的条件。作为特例,基于分位数选择或共形p值多重检验程序的SCOP在温和条件下具有有效覆盖保证。数值实验结果验证了SCOP在FCR控制中的有效性和鲁棒性,并表明其在多种场景下较现有方法能生成更窄的预测区间。