Conformal inference is a popular tool for constructing prediction intervals (PI). We consider here the scenario of post-selection/selective conformal inference, that is PIs are reported only for individuals selected from an unlabeled test data. To account for multiplicity, we develop a general split conformal framework to construct selective PIs with the false coverage-statement rate (FCR) control. We first investigate the Benjamini and Yekutieli (2005)'s FCR-adjusted method in the present setting, and show that it is able to achieve FCR control but yields uniformly inflated PIs. We then propose a novel solution to the problem, named as Selective COnditional conformal Predictions (SCOP), which entails performing selection procedures on both calibration set and test set and construct marginal conformal PIs on the selected sets by the aid of conditional empirical distribution obtained by the calibration set. Under a unified framework and exchangeable assumptions, we show that the SCOP can exactly control the FCR. More importantly, we provide non-asymptotic miscoverage bounds for a general class of selection procedures beyond exchangeablity and discuss the conditions under which the SCOP is able to control the FCR. As special cases, the SCOP with quantile-based selection or conformal p-values-based multiple testing procedures enjoys valid coverage guarantee under mild conditions. Numerical results confirm the effectiveness and robustness of SCOP in FCR control and show that it achieves more narrowed PIs over existing methods in many settings.
翻译:共形推断是构建预测区间(PI)的常用工具。本文考虑后选择/选择性共形推断场景,即仅对从无标签测试数据中选出的个体报告预测区间。为处理多重性问题,我们提出一个通用分割共形框架,在控制错误覆盖声明率(FCR)的同时构建选择性预测区间。首先研究Benjamini与Yekutieli(2005)的FCR校正方法在当前场景中的应用,证明其虽能实现FCR控制但会导致统一膨胀的预测区间。随后提出名为选择性条件共形预测(SCOP)的新型解决方案,该方案要求在标定集与测试集上同时执行选择程序,并借助标定集获得的经验条件分布为所选集构建边际共形预测区间。在统一框架与可交换性假设下,证明SCOP能精确控制FCR。更重要的是,我们为超越可交换性的通用选择程序类提供了非渐近误覆盖界,并讨论了SCOP能控制FCR的条件。作为特例,基于分位数选择或共形p值多重检验程序的SCOP在温和条件下具有有效的覆盖保证。数值结果证实了SCOP在FCR控制方面的有效性与鲁棒性,并表明其在多种场景下能比现有方法获得更窄的预测区间。