Conformal Test Martingales (CTMs) are a standard method within the Conformal Prediction framework for testing the crucial assumption of data exchangeability by monitoring deviations from uniformity in the p-value sequence. Although exchangeability implies uniform p-values, the converse does not hold. This raises the question of whether a significant break in exchangeability can occur, such that the p-values remain uniform, rendering CTMs blind. We answer this affirmatively, demonstrating the phenomenon of \emph{conformal blindness}. Through explicit construction, for the theoretically ideal ``oracle'' conformity measure (given by the true conditional density), we demonstrate the possibility of an \emph{$A$-cryptic change-point} (where $A$ refers to the conformity measure). Using bivariate Gaussian distributions, we identify a line along which a change in the marginal means does not alter the distribution of the conformity scores, thereby producing perfectly uniform p-values. Simulations confirm that even a massive distribution shift can be perfectly cryptic to the CTM, highlighting a fundamental limitation and emphasising the critical role of the alignment of the conformity measure with potential shifts.
翻译:共形测试鞅(CTMs)是共形预测框架内的一种标准方法,用于通过监测p值序列对均匀性的偏离来检验数据可交换性这一关键假设。尽管可交换性意味着均匀的p值,但反之并不成立。这就引出了一个疑问:是否可能发生显著的可交换性断裂,而p值却保持均匀,从而导致CTMs失效?我们对此给出了肯定回答,并证明了\emph{共形盲区}现象的存在。通过显式构造,针对理论理想的“先知”共形度量(由真实条件密度给出),我们证明了\emph{$A$-隐秘变点}(其中$A$指代共形度量)存在的可能性。利用二元高斯分布,我们识别出一条直线,沿该直线的边缘均值变化不会改变共形得分的分布,从而产生完全均匀的p值。仿真实验证实,即使发生巨大的分布偏移,对CTM而言仍可能是完全隐秘的,这揭示了一个根本性局限,并强调了共形度量与潜在偏移方向对齐的关键作用。