Conformal inference is a fundamental and versatile tool that provides distribution-free guarantees for many machine learning tasks. We consider the transductive setting, where decisions are made on a test sample of $m$ new points, giving rise to $m$ conformal $p$-values. {While classical results only concern their marginal distribution, we show that their joint distribution follows a P\'olya urn model, and establish a concentration inequality for their empirical distribution function.} The results hold for arbitrary exchangeable scores, including {\it adaptive} ones that can use the covariates of the test+calibration samples at training stage for increased accuracy. We demonstrate the usefulness of these theoretical results through uniform, in-probability guarantees for two machine learning tasks of current interest: interval prediction for transductive transfer learning and novelty detection based on two-class classification.
翻译:共形推理是一种基础且通用的工具,可为许多机器学习任务提供无分布假设的保证。我们考虑转导设定,即对包含 $m$ 个新点的测试样本做出决策,从而产生 $m$ 个共形 $p$ 值。虽然经典结果仅关注其边际分布,但我们证明其联合分布遵循 Pólya 瓮模型,并建立了其经验分布函数的浓度不等式。这些结果对任意可交换得分均成立,包括可在训练阶段利用测试+校准样本协变量以提高准确性的**自适应**得分。我们通过两个当前关注的机器学习任务(基于转导迁移学习的区间预测和基于二分类的新颖性检测)的均匀概率保证,展示了这些理论结果的有效性。