We study online conformal prediction for non-stationary data streams subject to unknown distribution drift. While most prior work studied this problem under adversarial settings and/or assessed performance in terms of gaps of time-averaged marginal coverage, we instead evaluate performance through training-conditional cumulative regret. We specifically focus on independently generated data with two types of distribution shift: abrupt change points and smooth drift. When non-conformity score functions are pretrained on an independent dataset, we propose a split-conformal style algorithm that leverages drift detection to adaptively update calibration sets, which provably achieves minimax-optimal regret. When non-conformity scores are instead trained online, we develop a full-conformal style algorithm that again incorporates drift detection to handle non-stationarity; this approach relies on stability - rather than permutation symmetry - of the model-fitting algorithm, which is often better suited to online learning under evolving environments. We establish non-asymptotic regret guarantees for our online full conformal algorithm, which match the minimax lower bound under appropriate restrictions on the prediction sets. Numerical experiments corroborate our theoretical findings.
翻译:本文研究面向未知分布漂移的非平稳数据流的在线共形预测问题。尽管已有研究多在对抗性设定下探讨该问题,且通过时间平均边缘覆盖率的差距评估性能,我们转而采用训练条件累积遗憾作为性能评估指标。我们特别关注两类分布漂移下独立生成的数据:突变点和渐进漂移。当非共形评分函数在独立数据集上预训练时,我们提出一种分割共形风格算法,该算法利用漂移检测自适应更新校准集,可证明达到极小极大最优遗憾。当非共形评分函数改为在线训练时,我们开发了一种全共形风格算法,该算法同样整合漂移检测以处理非平稳性;该方法依赖于模型拟合算法的稳定性(而非置换对称性),这通常更适用于动态环境下的在线学习。我们为在线全共形算法建立了非渐近遗憾保证,该保证在预测集的适当约束下与极小极大下界相匹配。数值实验验证了我们的理论发现。